Repository: dbt-labs/dbt-core

Issue:
[CT-258] [Spike] Upgrade to Jinja3?
Picking up from https://github.com/dbt-labs/dbt-core/issues/4745#issuecomment-1044354226:

> We've hesitated to do this, given the risk of breaking changes to user/project code—but we need to weigh that against the risk of running into this type of dependency/installation issue, given that Jinja2 is no longer officially supported and won't be patched accordingly.

Let's take the time to look through the [set of (breaking) changes](https://jinja.palletsprojects.com/en/3.0.x/changes/#version-3-0-0), and predict the impact on user/project code. Based on that analysis, we should move ahead with an upgrade, or consider our alternatives. Forking/vendoring `Jinja2` is far from ideal, but it is an option on the table.


Discussion:
@jtcohen6 did you run this with Jinja3?


Goal: What are our most viable options for leaving Jinja2? If so what is the impact or work that needs to be done? If moving moving to Jinja3, what is the plan? 

Potential Options:
1) Upgrade to Jinja3
2) Fork Jinja2
3) Own Jinja parser
4) Anything else you can think of
> @jtcohen6 did you run this with Jinja3?

- Our integration tests seem to be passing with `Jinja2==3.0.3`, it's just `mypy` that's failing: https://github.com/dbt-labs/dbt-core/pull/4262
- I've managed to successfully parse + run our internal analytics project with `Jinja2==3.0.3` installed
If it's simple to upgrade to jinja 3 we will likely do that. so that will be the first step of the spike. Outcome of this spike is a write up. write up includes consequences if we stay with jinja 2.
Hi guys, when can we expect the release supporting Jinja3? At the moment - I'm not able to create an environment for generating both mkdocs and dbt docs (we deploy them together), because there's a version conflict between `dbt-core` and `mkdocs-material` related to `jinja2`:
```
  Because no versions of dbt-redshift match >1.1.0,<2.0.0
   and dbt-redshift (1.1.0) depends on dbt-postgres (>=1.1.0,<1.2.0), dbt-redshift (>=1.1.0,<2.0.0) requires dbt-postgres (>=1.1.0,<1.2.0).
  And because no versions of dbt-postgres match >1.1.1,<1.2.0
   and dbt-postgres (1.1.1) depends on dbt-core (1.1.1), dbt-redshift (>=1.1.0,<2.0.0) requires dbt-core (1.1.1) or dbt-postgres (>=1.1.0,<1.1.1).
  And because dbt-core (1.1.1) depends on Jinja2 (2.11.3)
   and mkdocs-material (8.3.6) depends on jinja2 (>=3.0.2), if dbt-redshift (>=1.1.0,<2.0.0) and mkdocs-material (8.3.6) then dbt-postgres (>=1.1.0,<1.1.1).
  And because no versions of mkdocs-material match >8.3.6,<9.0.0
   and temp depends on mkdocs-material (^8.3.6), dbt-redshift (>=1.1.0,<2.0.0) requires dbt-postgres (>=1.1.0,<1.1.1).
  So, because temp depends on both dbt-postgres (^1.1.1) and dbt-redshift (^1.1.0), version solving failed.
```
As a workaround I have to create 2 separate venvs.
Same here

Resolving patch:
diff --git a/core/dbt/clients/jinja.py b/core/dbt/clients/jinja.py
--- a/core/dbt/clients/jinja.py
+++ b/core/dbt/clients/jinja.py
@@ -161,7 +161,7 @@ def quoted_native_concat(nodes):
 
 
 class NativeSandboxTemplate(jinja2.nativetypes.NativeTemplate):  # mypy: ignore
-    environment_class = NativeSandboxEnvironment
+    environment_class = NativeSandboxEnvironment  # type: ignore
 
     def render(self, *args, **kwargs):
         """Render the template to produce a native Python type. If the
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -48,7 +48,7 @@
         ],
     },
     install_requires=[
-        "Jinja2==2.11.3",
+        "Jinja2==3.1.2",
         "MarkupSafe>=0.23,<2.1",
         "agate>=1.6,<1.6.4",
         "click>=7.0,<9",


Test patch: