Repository: mitsuhiko/minijinja (language: rust)

Issue:
set_lstrip_blocks should not apply to expressions, only to blocks.
## Description

I've been using the newly added support for set_lstrip_blocks. Thank you for that! I've noticed, though, that i think it is also stripping leading spaces from expressions -- {{ }}. Those aren't blocks, right? If an expression is positioned with leading whitespace in the file, it should stay that way, I think.

Here's the way it works in python:

```
[16:52:32][kylederr@KylesPenalMBPM3][~/Code/derrley/testing_jinja]
$ python3 script.py
    one
    two

(venv)
[16:52:33][kylederr@KylesPenalMBPM3][~/Code/derrley/testing_jinja]
$ cat script.py
import jinja2
templateLoader = jinja2.FileSystemLoader(searchpath=".")
templateEnv = jinja2.Environment(
  loader=templateLoader,
  lstrip_blocks=True,
  trim_blocks=True
)
template = templateEnv.get_template('tmpl.tmp')
templateVars = {
  "things": [
    "one",
    "two",
  ]
}

print(template.render(templateVars))
(venv)
[16:52:48][kylederr@KylesPenalMBPM3][~/Code/derrley/testing_jinja]
$ cat tmpl.tmp
{% for thing in things %}
    {{ thing }}
{% endfor %}
```

## Reproduction steps

Do the above, in rust:

```
use minijinja::{context, path_loader, Environment};

fn main() {
    let mut env = Environment::new();
    env.set_trim_blocks(true);
    env.set_lstrip_blocks(true);
    env.set_loader(path_loader("."));
    let tmpl = env.get_template("tmpl.tmp").unwrap();
    println!(
        "{}",
        tmpl.render(context! {things => vec!["one", "two"]})
            .unwrap()
    )
}

```

and get this output:

```
$ cargo run --bin testtmpl
   Compiling rules v0.1.0 (/Users/kylederr/Code/perpetualsystems/simba/rules)
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
     Running `/Users/kylederr/Code/perpetualsystems/simba/target/debug/testtmpl`
one
two
```

(note they are not indented)

Additional helpful information:

- Version of minijinja: 1.0.20
- Version of rustc: rustc 1.76.0 (07dca489a 2024-02-04)
- Operating system and version: macos sonoma 14.2.1

## What did you expect

For the blocks to be indented, as per above.





Discussion / hints:

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 01384460..978b0bb5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,11 @@
 
 All notable changes to MiniJinja are documented here.
 
+## 1.0.21
+
+- Fixed an issue where `lstrip_blocks` unintentionally also applied to
+  variable expression blocks.  #502
+
 ## 1.0.20
 
 - Added support for implicit string concatenation for Jinja2 compatibility.  #489
diff --git a/minijinja/src/compiler/lexer.rs b/minijinja/src/compiler/lexer.rs
index e535527b..cb413689 100644
--- a/minijinja/src/compiler/lexer.rs
+++ b/minijinja/src/compiler/lexer.rs
@@ -491,6 +491,7 @@ impl<'s> Tokenizer<'s> {
         }
     }
 
+    syntax_token_getter!(variable_start, "{{");
     syntax_token_getter!(variable_end, "}}");
     syntax_token_getter!(block_start, "{%");
     syntax_token_getter!(block_end, "%}");
@@ -506,7 +507,11 @@ impl<'s> Tokenizer<'s> {
         }
         let old_loc = self.loc();
         let (lead, span) = match find_start_marker(self.rest, &self.syntax_config) {
-            Some((start, Whitespace::Default)) if self.ws_config.lstrip_blocks => {
+            Some((start, Whitespace::Default))
+                if self.ws_config.lstrip_blocks
+                    && self.rest.get(start..start + self.variable_start().len())
+                        != Some(self.variable_start()) =>
+            {
                 let peeked = &self.rest[..start];
                 let trimmed = lstrip_block(peeked);
                 let lead = self.advance(trimmed.len());


Test patch:
diff --git a/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks-preserve.txt.snap b/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks-preserve.txt.snap
index 6e88b380..8c3446c0 100644
--- a/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks-preserve.txt.snap
+++ b/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks-preserve.txt.snap
@@ -47,8 +47,8 @@ Ident("seq")
   "seq"
 BlockEnd
   "+%}"
-TemplateData("\n")
-  "\n"
+TemplateData("\n    ")
+  "\n    "
 VariableStart
   "{{"
 Ident("item")
@@ -65,4 +65,3 @@ BlockEnd
   "+%}"
 TemplateData("\n</ul>")
   "\n</ul>"
-
diff --git a/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks.txt.snap b/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks.txt.snap
index f5241b4f..84f43eb3 100644
--- a/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks.txt.snap
+++ b/minijinja/tests/snapshots/test_lexer__lexer@lstrip-blocks.txt.snap
@@ -47,6 +47,8 @@ Ident("seq")
   "seq"
 BlockEnd
   "%}"
+TemplateData("    ")
+  "    "
 VariableStart
   "{{"
 Ident("item")
@@ -63,4 +65,3 @@ BlockEnd
   "%}"
 TemplateData("</ul>")
   "</ul>"
-