Repository: sqlfluff/sqlfluff

Issue:
sqlfluff doesn't recognise a jinja variable set inside of "if" statement
### Search before asking

- [X] I searched the [issues](https://github.com/sqlfluff/sqlfluff/issues) and found no similar issues.


### What Happened

When I try to define a jinja variable using "set" jinja directive inside of an "if" jinja statement, sqlfluff complains: 
"Undefined jinja template variable".

### Expected Behaviour

to not have a linting issue

### Observed Behaviour

sqlfluff lint gives an error:
"Undefined jinja template variable"

### How to reproduce

try to create a "temp.sql" file with the following content

```
{% if True %}
    {% set some_var %}1{% endset %}
    SELECT {{some_var}}
{% endif %}
```

and run:
```
sqlfluff lint ./temp.sql
```

You will get the following error:
```
== [./temp.sql] FAIL                                                                                                                    
L:   2 | P:  12 |  TMP | Undefined jinja template variable: 'some_var'
L:   3 | P:  14 |  TMP | Undefined jinja template variable: 'some_var'
```

### Dialect

tested on 'snowflake' dialect

### Version

sqlfluff, version 0.11.1
Python 3.8.12

### Configuration

[sqlfluff]
verbose = 1
dialect = snowflake
templater = jinja
exclude_rules = L027,L031,L032,L036,L044,L046,L034,L050
output_line_length = 121
sql_file_exts=.sql

[sqlfluff:rules]
tab_space_size = 4
max_line_length = 250
indent_unit = space
comma_style = trailing
allow_scalar = True
single_table_references = consistent
unquoted_identifiers_policy = aliases

[sqlfluff:rules:L042]
forbid_subquery_in = both

[sqlfluff:rules:L010]  # Keywords
capitalisation_policy = upper

[sqlfluff:rules:L014]
extended_capitalisation_policy = lower

[sqlfluff:rules:L030]  # function names
extended_capitalisation_policy = upper

### Are you willing to work on and submit a PR to address the issue?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/sqlfluff/sqlfluff/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Does Jinja support this? I don't see how this could be a SQLFluff issue.
Your example does work on this website. I wonder if there's a Jinja runtime setting that affects whether this works.

http://jinja.quantprogramming.com/
It also works with `j2cli` on my local machine. Seems like this _has_ to be a Jinja runtime setting...

https://github.com/kolypto/j2cli
It was added in Jinja 2.8: https://jinja.palletsprojects.com/en/3.0.x/templates/#block-assignments

Not sure what version we pull in depending on our other dependencies?

I'm digging into this more. SQLFluff contains some additional code that attempts to detect undeclared Jinja variables and provide better error handling. The "issue" is being detected and reported by that code, not by Jinja itself. So we should be able to fix this. Need to do this carefully so we don't break error reporting for real errors. 
I think I have a fix. Just need to make the undefined variable check more sophisticated.

Patch:
diff --git a/src/sqlfluff/core/templaters/jinja.py b/src/sqlfluff/core/templaters/jinja.py
--- a/src/sqlfluff/core/templaters/jinja.py
+++ b/src/sqlfluff/core/templaters/jinja.py
@@ -343,20 +343,33 @@ def process(
         # first Exception which serves only to catch catastrophic errors.
         try:
             syntax_tree = env.parse(in_str)
-            undefined_variables = meta.find_undeclared_variables(syntax_tree)
+            potentially_undefined_variables = meta.find_undeclared_variables(
+                syntax_tree
+            )
         except Exception as err:  # pragma: no cover
             # TODO: Add a url here so people can get more help.
             raise SQLTemplaterError(f"Failure in identifying Jinja variables: {err}.")
 
-        # Get rid of any that *are* actually defined.
-        for val in live_context:
-            if val in undefined_variables:
-                undefined_variables.remove(val)
+        undefined_variables = set()
+
+        class Undefined:
+            """Similar to jinja2.StrictUndefined, but remembers, not fails."""
+
+            def __init__(self, name):
+                self.name = name
+
+            def __str__(self):
+                """Treat undefined vars as empty, but remember for later."""
+                undefined_variables.add(self.name)
+                return ""
+
+            def __getattr__(self, item):
+                undefined_variables.add(self.name)
+                return Undefined(f"{self.name}.{item}")
 
-        if undefined_variables:
-            # Lets go through and find out where they are:
-            for val in self._crawl_tree(syntax_tree, undefined_variables, in_str):
-                violations.append(val)
+        for val in potentially_undefined_variables:
+            if val not in live_context:
+                live_context[val] = Undefined(name=val)
 
         try:
             # NB: Passing no context. Everything is loaded when the template is loaded.
@@ -368,6 +381,10 @@ def process(
                 config=config,
                 make_template=make_template,
             )
+            if undefined_variables:
+                # Lets go through and find out where they are:
+                for val in self._crawl_tree(syntax_tree, undefined_variables, in_str):
+                    violations.append(val)
             return (
                 TemplatedFile(
                     source_str=in_str,


Test patch:
diff --git a/test/core/templaters/jinja_test.py b/test/core/templaters/jinja_test.py
--- a/test/core/templaters/jinja_test.py
+++ b/test/core/templaters/jinja_test.py
@@ -411,6 +411,20 @@ def test__templater_jinja_error_variable():
     assert any(v.rule_code() == "TMP" and v.line_no == 1 for v in vs)
 
 
+def test__templater_jinja_dynamic_variable_no_violations():
+    """Test no templater violation for variable defined within template."""
+    t = JinjaTemplater(override_context=dict(blah="foo"))
+    instr = """{% if True %}
+    {% set some_var %}1{% endset %}
+    SELECT {{some_var}}
+{% endif %}
+"""
+    outstr, vs = t.process(in_str=instr, fname="test", config=FluffConfig())
+    assert str(outstr) == "\n    \n    SELECT 1\n\n"
+    # Check we have no violations.
+    assert len(vs) == 0
+
+
 def test__templater_jinja_error_syntax():
     """Test syntax problems in the jinja templater."""
     t = JinjaTemplater()