Repository: dbt-labs/dbt-core

Issue:
0.17.0 regression: Make native env rendering explicit/opt-in
### Describe the bug
See the discussion in https://github.com/fishtown-analytics/dbt/pull/2608 for more information.

dbt v0.17.0 adds a native env renderer for Jinja. This has the _great_ benefit of coercing stringy values like `"True"` or `"False"` to boolean values like `True` and `False`. It's a subtle distinction, and the benefit is that you can use dbt's jinja context to do really clever things in configuration files, eg:

```
models:
  my_package:
    optional:
       enabled: "{{ env_var('DBT_OPTIONAL_MODELS_ENABLED") }}
```

In this example, the `env_var` function returns a string, but with the native environment, dbt coerces the string to a boolean value. This is necessary because the `enabled` config requires a boolean value as a configuration.

The drawbacks of this approach are equally subtle, but there are many:
 - numeric strings can be errantly converted into integers, losing information (eg. `"00"` --> `0`)
 - confusingly, this behavior is inconsistent with the current implementation for obscure reasons (`"01"` --> `"01"`)*
 - values like the empty string (`''`) are evaluated to `None` without special handling
 - quotes in quoted strings are stripped as a part of literal evaluation (eg. `"'abc'"` --> `"abc"`)
 - the literal evaluation lib that dbt uses will coerce "container" types (eg. `"[1,2,3]"` --> `[1,2,3]` as a list instead of a string)

* `01` is not a valid Python literal, so a literal evaluation of the string raises a `SyntaxError` and dbt returns the source string instead

So, this feature is powerful and compelling (which is why we added it in the first place!) but it frequently results in unexpected behavior if you don't know the gritty details about how dbt's native env rendering works under the hood. That's the exact opposite of our design goals as we march towards a 1.0 release of dbt, so we unfortunately need to take some action to revert the native env changes introduced in 0.17.0 and gate them behind a syntax which makes their use explicit.

This is going to be a breaking change to a breaking change, so we'll need to take care to communicate out why we're changing this and how users will need to adjust their code.

Example code that should work once this issue is addressed:
```
models:
  my_package:
    optional:
       enabled: "{{ env_var('DBT_OPTIONAL_MODELS_ENABLED") | as_bool }}
```

### The path forwards:

Add a `NativeMarker` as described in this comment (https://github.com/fishtown-analytics/dbt/pull/2608#issuecomment-654438650) which can be accessed using an `as_native`. Default to _not_ calling `literal_eval` on strings unless the `as_native` marker is provided.

The `as_native` filter is good and useful in many cases, but we should also add some sugary filters that are more explicit. `as_native` isn't super descriptive, and we should take care to make sure that this behavior is well defined and well documented. I propose:

- `<expression> | as_text`: same behavior as 0.17.0 (and more or less a no-op in dbt > 0.17.0)
- `<expression> | as_bool`: coerce the string to a boolean value. Fail if the input cannot be represented as a bool
- `<expression> | as_number`: coerce the string to a number value. Fail if the input cannot be represented as a number (or, do we want `as_int` / `as_float` here instead?)
- `<expression> | as_native`: coerce the string into its native representation according to `ast.literal_eval`. That could be a set/list/tuple/dict/etc, and it's on the user to make sure that their values inside the container are evaluated correctly. From my understanding, there isn't _really_ a good way to make sure dbt does the right thing there 100% of the time, which is kind of the crux of this issue!

This might require the addition of some additional Marker classes, but I think there will be a ton of benefit to coupling type inference and validation in the same jinja filter.

### Next steps

- Drew to write something up since functionality churn like this is a little unusual and is blocking our dbt patch/bugfix release
- We need to determine the version number for the release that includes this change. 0.17.1 and we add warning labels to 0.17.0? Or 0.18.0 and we acknowledge the breaking change as such? TBD
- Drew to write up lots of docs and some sort of discourse post outlining the whats and the whys


Discussion:

Resolving patch:
diff --git a/core/dbt/clients/jinja.py b/core/dbt/clients/jinja.py
--- a/core/dbt/clients/jinja.py
+++ b/core/dbt/clients/jinja.py
@@ -8,7 +8,8 @@
 from contextlib import contextmanager
 from itertools import chain, islice
 from typing import (
-    List, Union, Set, Optional, Dict, Any, Iterator, Type, NoReturn, Tuple
+    List, Union, Set, Optional, Dict, Any, Iterator, Type, NoReturn, Tuple,
+    Callable
 )
 
 import jinja2
@@ -28,7 +29,7 @@
 from dbt.contracts.graph.parsed import ParsedSchemaTestNode
 from dbt.exceptions import (
     InternalException, raise_compiler_error, CompilationException,
-    invalid_materialization_argument, MacroReturn
+    invalid_materialization_argument, MacroReturn, JinjaRenderingException
 )
 from dbt.flags import MACRO_DEBUGGING
 from dbt.logger import GLOBAL_LOGGER as logger  # noqa
@@ -111,6 +112,24 @@ class TextMarker(str):
     """
 
 
+class NativeMarker(str):
+    """A special native-env marker that indicates the field should be passed to
+    literal_eval.
+    """
+
+
+class BoolMarker(NativeMarker):
+    pass
+
+
+class NumberMarker(NativeMarker):
+    pass
+
+
+def _is_number(value) -> bool:
+    return isinstance(value, (int, float)) and not isinstance(value, bool)
+
+
 def quoted_native_concat(nodes):
     """This is almost native_concat from the NativeTemplate, except in the
     special case of a single argument that is a quoted string and returns a
@@ -119,23 +138,31 @@ def quoted_native_concat(nodes):
     head = list(islice(nodes, 2))
 
     if not head:
-        return None
+        return ''
 
     if len(head) == 1:
         raw = head[0]
         if isinstance(raw, TextMarker):
             return str(raw)
+        elif not isinstance(raw, NativeMarker):
+            # return non-strings as-is
+            return raw
     else:
-        raw = "".join([str(v) for v in chain(head, nodes)])
+        # multiple nodes become a string.
+        return "".join([str(v) for v in chain(head, nodes)])
 
     try:
         result = literal_eval(raw)
     except (ValueError, SyntaxError, MemoryError):
-        return raw
-
-    # if it was a str and it still is a str, return it as-is.
-    if isinstance(result, str):
         result = raw
+    if isinstance(raw, BoolMarker) and not isinstance(result, bool):
+        raise JinjaRenderingException(
+            f"Could not convert value '{raw!s}' into type 'bool'"
+        )
+    if isinstance(raw, NumberMarker) and not _is_number(result):
+        raise JinjaRenderingException(
+            f"Could not convert value '{raw!s}' into type 'number'"
+        )
 
     return result
 
@@ -417,6 +444,22 @@ def __reduce__(self):
     return Undefined
 
 
+NATIVE_FILTERS: Dict[str, Callable[[Any], Any]] = {
+    'as_text': TextMarker,
+    'as_bool': BoolMarker,
+    'as_native': NativeMarker,
+    'as_number': NumberMarker,
+}
+
+
+TEXT_FILTERS: Dict[str, Callable[[Any], Any]] = {
+    'as_text': lambda x: x,
+    'as_bool': lambda x: x,
+    'as_native': lambda x: x,
+    'as_number': lambda x: x,
+}
+
+
 def get_environment(
     node=None,
     capture_macros: bool = False,
@@ -436,13 +479,13 @@ def get_environment(
     text_filter: Type
     if native:
         env_cls = NativeSandboxEnvironment
-        text_filter = TextMarker
+        filters = NATIVE_FILTERS
     else:
         env_cls = MacroFuzzEnvironment
-        text_filter = str
+        filters = TEXT_FILTERS
 
     env = env_cls(**args)
-    env.filters['as_text'] = text_filter
+    env.filters.update(filters)
 
     return env
 
diff --git a/core/dbt/config/profile.py b/core/dbt/config/profile.py
--- a/core/dbt/config/profile.py
+++ b/core/dbt/config/profile.py
@@ -8,6 +8,7 @@
 from dbt.clients.yaml_helper import load_yaml_text
 from dbt.contracts.connection import Credentials, HasCredentials
 from dbt.contracts.project import ProfileConfig, UserConfig
+from dbt.exceptions import CompilationException
 from dbt.exceptions import DbtProfileError
 from dbt.exceptions import DbtProjectError
 from dbt.exceptions import ValidationException
@@ -268,7 +269,10 @@ def render_profile(
             raw_profile, profile_name, target_name
         )
 
-        profile_data = renderer.render_data(raw_profile_data)
+        try:
+            profile_data = renderer.render_data(raw_profile_data)
+        except CompilationException as exc:
+            raise DbtProfileError(str(exc)) from exc
         return target_name, profile_data
 
     @classmethod
diff --git a/core/dbt/exceptions.py b/core/dbt/exceptions.py
--- a/core/dbt/exceptions.py
+++ b/core/dbt/exceptions.py
@@ -256,6 +256,10 @@ def __reduce__(self):
         return (JSONValidationException, (self.typename, self.errors))
 
 
+class JinjaRenderingException(CompilationException):
+    pass
+
+
 class UnknownAsyncIDException(Exception):
     CODE = 10012
     MESSAGE = 'RPC server got an unknown async ID'


Test patch:
diff --git a/test/integration/039_config_test/test_configs.py b/test/integration/039_config_test/test_configs.py
--- a/test/integration/039_config_test/test_configs.py
+++ b/test/integration/039_config_test/test_configs.py
@@ -110,9 +110,9 @@ def postgres_profile(self):
                     'default2': {
                         'type': 'postgres',
                         # make sure you can do this and get an int out
-                        'threads': "{{ 1 + 3 }}",
+                        'threads': "{{ (1 + 3) | as_number }}",
                         'host': self.database_host,
-                        'port': "{{ 5400 + 32 }}",
+                        'port': "{{ (5400 + 32) | as_number }}",
                         'user': 'root',
                         'pass': 'password',
                         'dbname': 'dbt',
@@ -121,9 +121,9 @@ def postgres_profile(self):
                     'disabled': {
                         'type': 'postgres',
                         # make sure you can do this and get an int out
-                        'threads': "{{ 1 + 3 }}",
+                        'threads': "{{ (1 + 3) | as_number }}",
                         'host': self.database_host,
-                        'port': "{{ 5400 + 32 }}",
+                        'port': "{{ (5400 + 32) | as_number }}",
                         'user': 'root',
                         'pass': 'password',
                         'dbname': 'dbt',
@@ -141,7 +141,7 @@ def project_config(self):
             'data-paths': ['data'],
             'models': {
                 'test': {
-                    'enabled': "{{ target.name == 'default2' }}",
+                    'enabled': "{{ (target.name == 'default2' | as_bool) }}",
                 },
             },
             # set the `var` result in schema.yml to be 'seed', so that the
@@ -155,7 +155,7 @@ def project_config(self):
                 'quote_columns': False,
                 'test': {
                     'seed': {
-                        'enabled': "{{ target.name == 'default2' }}",
+                        'enabled': "{{ (target.name == 'default2') | as_bool }}",
                     },
                 },
             },
diff --git a/test/integration/base.py b/test/integration/base.py
--- a/test/integration/base.py
+++ b/test/integration/base.py
@@ -184,7 +184,7 @@ def redshift_profile(self):
                         'type': 'redshift',
                         'threads': 1,
                         'host': os.getenv('REDSHIFT_TEST_HOST'),
-                        'port': os.getenv('REDSHIFT_TEST_PORT'),
+                        'port': int(os.getenv('REDSHIFT_TEST_PORT')),
                         'user': os.getenv('REDSHIFT_TEST_USER'),
                         'pass': os.getenv('REDSHIFT_TEST_PASS'),
                         'dbname': os.getenv('REDSHIFT_TEST_DBNAME'),
diff --git a/test/unit/test_config.py b/test/unit/test_config.py
--- a/test/unit/test_config.py
+++ b/test/unit/test_config.py
@@ -131,7 +131,7 @@ def setUp(self):
                     'with-vars': {
                         'type': "{{ env_var('env_value_type') }}",
                         'host': "{{ env_var('env_value_host') }}",
-                        'port': "{{ env_var('env_value_port') }}",
+                        'port': "{{ env_var('env_value_port') | as_number }}",
                         'user': "{{ env_var('env_value_user') }}",
                         'pass': "{{ env_var('env_value_pass') }}",
                         'dbname': "{{ env_var('env_value_dbname') }}",
@@ -140,7 +140,7 @@ def setUp(self):
                     'cli-and-env-vars': {
                         'type': "{{ env_var('env_value_type') }}",
                         'host': "{{ var('cli_value_host') }}",
-                        'port': "{{ env_var('env_value_port') }}",
+                        'port': "{{ env_var('env_value_port') | as_number }}",
                         'user': "{{ env_var('env_value_user') }}",
                         'pass': "{{ env_var('env_value_pass') }}",
                         'dbname': "{{ env_var('env_value_dbname') }}",
@@ -367,7 +367,7 @@ def test_invalid_env_vars(self):
                     renderer,
                     target_override='with-vars'
                 )
-        self.assertIn("not of type 'integer'", str(exc.exception))
+        self.assertIn("Could not convert value 'hello' into type 'number'", str(exc.exception))
 
 
 class TestProfileFile(BaseFileTest):
@@ -511,7 +511,7 @@ def test_invalid_env_vars(self):
             with self.assertRaises(dbt.exceptions.DbtProfileError) as exc:
                 self.from_args()
 
-        self.assertIn("not of type 'integer'", str(exc.exception))
+        self.assertIn("Could not convert value 'hello' into type 'number'", str(exc.exception))
 
     def test_cli_and_env_vars(self):
         self.args.target = 'cli-and-env-vars'
diff --git a/test/unit/test_jinja.py b/test/unit/test_jinja.py
--- a/test/unit/test_jinja.py
+++ b/test/unit/test_jinja.py
@@ -1,3 +1,4 @@
+from contextlib import contextmanager
 import pytest
 import unittest
 import yaml
@@ -5,7 +6,373 @@
 from dbt.clients.jinja import get_rendered
 from dbt.clients.jinja import get_template
 from dbt.clients.jinja import extract_toplevel_blocks
-from dbt.exceptions import CompilationException
+from dbt.exceptions import CompilationException, JinjaRenderingException
+
+
+@contextmanager
+def returns(value):
+    yield value
+
+
+@contextmanager
+def raises(value):
+    with pytest.raises(value) as exc:
+        yield exc
+
+
+def expected_id(arg):
+    if isinstance(arg, list):
+        return '_'.join(arg)
+
+
+jinja_tests = [
+    # strings
+    (
+        '''foo: bar''',
+        returns('bar'),
+        returns('bar'),
+    ),
+    (
+        '''foo: "bar"''',
+        returns('bar'),
+        returns('bar'),
+    ),
+    (
+        '''foo: "'bar'"''',
+        returns("'bar'"),
+        returns("'bar'"),
+    ),
+    (
+        """foo: '"bar"'""",
+        returns('"bar"'),
+        returns('"bar"'),
+    ),
+    (
+        '''foo: "{{ 'bar' | as_text }}"''',
+        returns('bar'),
+        returns('bar'),
+    ),
+    (
+        '''foo: "{{ 'bar' | as_bool }}"''',
+        returns('bar'),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ 'bar' | as_number }}"''',
+        returns('bar'),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ 'bar' | as_native }}"''',
+        returns('bar'),
+        returns('bar'),
+    ),
+    # ints
+    (
+        '''foo: 1''',
+        returns('1'),
+        returns('1'),
+    ),
+    (
+        '''foo: "1"''',
+        returns('1'),
+        returns('1'),
+    ),
+    (
+        '''foo: "'1'"''',
+        returns("'1'"),
+        returns("'1'"),
+    ),
+    (
+        """foo: '"1"'""",
+        returns('"1"'),
+        returns('"1"'),
+    ),
+    (
+        '''foo: "{{ 1 }}"''',
+        returns('1'),
+        returns('1'),
+    ),
+    (
+        '''foo: "{{ '1' }}"''',
+        returns('1'),
+        returns('1'),
+    ),
+    (
+        '''foo: "'{{ 1 }}'"''',
+        returns("'1'"),
+        returns("'1'"),
+    ),
+    (
+        '''foo: "'{{ '1' }}'"''',
+        returns("'1'"),
+        returns("'1'"),
+    ),
+    (
+        '''foo: "{{ 1 | as_text }}"''',
+        returns('1'),
+        returns('1'),
+    ),
+    (
+        '''foo: "{{ 1 | as_bool }}"''',
+        returns('1'),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ 1 | as_number }}"''',
+        returns('1'),
+        returns(1),
+    ),
+    (
+        '''foo: "{{ 1 | as_native }}"''',
+        returns('1'),
+        returns(1),
+    ),
+    (
+        '''foo: "{{ '1' | as_text }}"''',
+        returns('1'),
+        returns('1'),
+    ),
+    (
+        '''foo: "{{ '1' | as_bool }}"''',
+        returns('1'),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ '1' | as_number }}"''',
+        returns('1'),
+        returns(1),
+    ),
+    (
+        '''foo: "{{ '1' | as_native }}"''',
+        returns('1'),
+        returns(1),
+    ),
+    # booleans.
+    # Note the discrepancy with true vs True: `true` is recognized by jinja but
+    # not literal_eval, but `True` is recognized by ast.literal_eval.
+    # For extra fun, yaml recognizes both.
+    # unquoted true
+    (
+        '''foo: "{{ True }}"''',
+        returns('True'),
+        returns('True'),
+    ),
+    (
+        '''foo: "{{ True | as_text }}"''',
+        returns('True'),
+        returns('True'),
+    ),
+    (
+        '''foo: "{{ True | as_bool }}"''',
+        returns('True'),
+        returns(True),
+    ),
+    (
+        '''foo: "{{ True | as_number }}"''',
+        returns('True'),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ True | as_native }}"''',
+        returns('True'),
+        returns(True),
+    ),
+    # unquoted true
+    (
+        '''foo: "{{ true }}"''',
+        returns("True"),
+        returns("True"),
+    ),
+    (
+        '''foo: "{{ true | as_text }}"''',
+        returns("True"),
+        returns("True"),
+    ),
+    (
+        '''foo: "{{ true | as_bool }}"''',
+        returns("True"),
+        returns(True),
+    ),
+    (
+        '''foo: "{{ true | as_number }}"''',
+        returns("True"),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ true | as_native }}"''',
+        returns("True"),
+        returns(True),
+    ),
+    (
+        '''foo: "{{ 'true' | as_text }}"''',
+        returns("true"),
+        returns("true"),
+    ),
+    # quoted 'true'
+    (
+        '''foo: "'{{ true }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),  # jinja true -> python True -> str(True) -> "True" -> quoted
+    (
+        '''foo: "'{{ true | as_text }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),
+    (
+        '''foo: "'{{ true | as_bool }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),
+    (
+        '''foo: "'{{ true | as_number }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),
+    (
+        '''foo: "'{{ true | as_native }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),
+    # unquoted True
+    (
+        '''foo: "{{ True }}"''',
+        returns('True'),
+        returns('True'),
+    ),
+    (
+        '''foo: "{{ True | as_text }}"''',
+        returns("True"),
+        returns("True"),
+    ),  # True -> string 'True' -> text -> str('True') -> 'True'
+    (
+        '''foo: "{{ True | as_bool }}"''',
+        returns("True"),
+        returns(True),
+    ),
+    (
+        '''foo: "{{ True | as_number }}"''',
+        returns("True"),
+        raises(JinjaRenderingException),
+    ),
+    (
+        '''foo: "{{ True | as_native }}"''',
+        returns("True"),
+        returns(True),
+    ),
+    # quoted 'True' within rendering
+    (
+        '''foo: "{{ 'True' | as_text }}"''',
+        returns("True"),
+        returns("True"),
+    ),
+    # 'True' -> string 'True' -> text -> str('True') -> 'True'
+    (
+        '''foo: "{{ 'True' | as_bool }}"''',
+        returns('True'),
+        returns(True),
+    ),
+    # quoted 'True' outside rendering
+    (
+        '''foo: "'{{ True }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),
+    (
+        '''foo: "'{{ True | as_bool }}'"''',
+        returns("'True'"),
+        returns("'True'"),
+    ),
+    # yaml turns 'yes' into a boolean true
+    (
+        '''foo: yes''',
+        returns('True'),
+        returns('True'),
+    ),
+    (
+        '''foo: "yes"''',
+        returns('yes'),
+        returns('yes'),
+    ),
+    # concatenation
+    (
+        '''foo: "{{ (a_int + 100) | as_native }}"''',
+        returns('200'),
+        returns(200),
+    ),
+    (
+        '''foo: "{{ (a_str ~ 100) | as_native }}"''',
+        returns('100100'),
+        returns(100100),
+    ),
+    (
+        '''foo: "{{( a_int ~ 100) | as_native }}"''',
+        returns('100100'),
+        returns(100100),
+    ),
+    # multiple nodes -> always str
+    (
+        '''foo: "{{ a_str | as_native }}{{ a_str | as_native }}"''',
+        returns('100100'),
+        returns('100100'),
+    ),
+    (
+        '''foo: "{{ a_int | as_native }}{{ a_int | as_native }}"''',
+        returns('100100'),
+        returns('100100'),
+    ),
+    (
+        '''foo: "'{{ a_int | as_native }}{{ a_int | as_native }}'"''',
+        returns("'100100'"),
+        returns("'100100'"),
+    ),
+    (
+        '''foo:''',
+        returns('None'),
+        returns('None'),
+    ),
+    (
+        '''foo: null''',
+        returns('None'),
+        returns('None'),
+    ),
+    (
+        '''foo: ""''',
+        returns(''),
+        returns(''),
+    ),
+    (
+        '''foo: "{{ '' | as_native }}"''',
+        returns(''),
+        returns(''),
+    ),
+    # very annoying, but jinja 'none' is yaml 'null'.
+    (
+        '''foo: "{{ none | as_native }}"''',
+        returns('None'),
+        returns(None),
+    ),
+]
+
+
+@pytest.mark.parametrize(
+    'value,text_expectation,native_expectation',
+    jinja_tests,
+    ids=expected_id
+)
+def test_jinja_rendering(value, text_expectation, native_expectation):
+    foo_value = yaml.safe_load(value)['foo']
+    ctx = {
+        'a_str': '100',
+        'a_int': 100,
+        'b_str': 'hello'
+    }
+    with text_expectation as text_result:
+        assert text_result == get_rendered(foo_value, ctx, native=False)
+
+    with native_expectation as native_result:
+        assert native_result == get_rendered(foo_value, ctx, native=True)
 
 
 class TestJinja(unittest.TestCase):
@@ -17,25 +384,25 @@ def test_do(self):
         self.assertEqual(mod.my_dict, {'a': 1})
 
     def test_regular_render(self):
-        s = '{{ "some_value" }}'
+        s = '{{ "some_value" | as_native }}'
         value = get_rendered(s, {}, native=False)
         assert value == 'some_value'
-        s = '{{ 1991 }}'
+        s = '{{ 1991 | as_native }}'
         value = get_rendered(s, {}, native=False)
         assert value == '1991'
 
         s = '{{ "some_value" | as_text }}'
-        value = get_rendered(s, {}, native=True)
+        value = get_rendered(s, {}, native=False)
         assert value == 'some_value'
         s = '{{ 1991 | as_text }}'
-        value = get_rendered(s, {}, native=True)
+        value = get_rendered(s, {}, native=False)
         assert value == '1991'
 
     def test_native_render(self):
-        s = '{{ "some_value" }}'
+        s = '{{ "some_value" | as_native }}'
         value = get_rendered(s, {}, native=True)
         assert value == 'some_value'
-        s = '{{ 1991 }}'
+        s = '{{ 1991 | as_native }}'
         value = get_rendered(s, {}, native=True)
         assert value == 1991
 
@@ -413,65 +780,3 @@ def test_if_endfor_newlines(self):
 hi
 {% endmaterialization %}
 '''
-
-
-native_expected_behaviors = [
-    # strings
-    ('''foo: bar''', 'bar'),
-    ('''foo: "bar"''', 'bar'),
-    ('''foo: "'bar'"''', "'bar'"),
-    ("""foo: '"bar"'""", '"bar"'),
-    # ints
-    ('''foo: 1''', 1),
-    ('''foo: "1"''', 1),
-    ('''foo: "'1'"''', "'1'"),
-    ('''foo: "{{ 1 }}"''', 1),
-    ('''foo: "{{ '1' }}"''', 1),
-    ('''foo: "'{{ 1 }}'"''', "'1'"),
-    ('''foo: "'{{ '1' }}'"''', "'1'"),
-    ('''foo: "{{ 1 | as_text }}"''', '1'),
-    ('''foo: "{{ '1' | as_text }}"''', '1'),
-    # booleans.
-    # Note the discrepancy with true vs True: `true` is recognized by jinja but
-    # not literal_eval, but `True` is recognized by ast.literal_eval.
-    # For extra fun, yaml recognizes both.
-    ('''foo: "{{ true }}"''', True),
-    ('''foo: "{{ 'true' }}"''', 'true'),
-    ('''foo: "'{{ true }}'"''', "'True'"),
-    ('''foo: "{{ true | as_text }}"''', "True"),  # true -> boolean True -> text -> str(True) -> 'True'
-    ('''foo: "{{ 'true' | as_text }}"''', "true"),  # 'true' -> string 'true' -> text -> str('true') -> 'true'
-    ('''foo: "{{ True }}"''', True),
-    ('''foo: "{{ 'True' }}"''', True),
-    ('''foo: "'{{ True }}'"''', "'True'"),
-    ('''foo: "{{ True | as_text }}"''', "True"),  # True -> string 'True' -> text -> str('True') -> 'True'
-    ('''foo: "{{ 'True' | as_text }}"''', "True"),  # 'True' -> string 'True' -> text -> str('True') -> 'True'
-    ('''foo: yes''', True),  # yaml turns 'yes' into a boolean true
-    ('''foo: "yes"''', "yes"),
-    # concatenation
-    ('''foo: "{{ a_int + 100 }}"''', 200),
-    ('''foo: "{{ a_str ~ 100 }}"''', 100100),
-    ('''foo: "{{ a_int ~ 100 }}"''', 100100),
-    ('''foo: "{{ a_str }}{{ a_str }}"''', 100100),
-    ('''foo: "{{ a_int }}{{ a_int }}"''', 100100),
-    ('''foo: "'{{ a_int }}{{ a_int }}'"''', "'100100'"),
-
-]
-
-
-def expected_id(arg):
-    if isinstance(arg, list):
-        return '_'.join(arg)
-
-
-@pytest.mark.parametrize(
-    'inputvalue,expected', native_expected_behaviors, ids=expected_id
-)
-def test_native_rendering(inputvalue, expected):
-    # this test is pretty useless without preprocessing things in yaml.
-    value = yaml.safe_load(inputvalue)['foo']
-    ctx = {
-        'a_str': '100',
-        'a_int': 100,
-        'b_str': 'hello'
-    }
-    assert get_rendered(value, ctx, native=True) == expected