Repository: ast-grep/ast-grep (language: rust)

Issue:
[bug] Some Kotlin pattern works differently between CLI and Playground
Thank you for creating a great tool! In the course of using it, I found a strange phenomenon where the behavior is different between CLI and Playground, and I would like to report it.

<!-- Please fill in each section if possible. Screenshot is welcome. Thank you! -->

# ⏯ Playground Link

<!--
  A link to a ast-grep Playground "Share" link which shows this behavior.
  This should have the same code as the code snippet below.
  As a last resort, you can link to a repo, but these will be slower for us to investigate.
-->
[Playground link with relevant code](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImtvdGxpbiIsInF1ZXJ5IjoiY29uc29sZS5sb2coJE1BVENIKSIsInJld3JpdGUiOiJsb2dnZXIubG9nKCRNQVRDSCkiLCJjb25maWciOiJpZDogcGFyYW1ldGVyLWluLXNlcmlhbGl6YWJsZS1jbGFzc1xubGFuZ3VhZ2U6IGtvdGxpblxucnVsZTpcbiAgcGF0dGVybjpcbiAgICBjb250ZXh0OiB8XG4gICAgICBjbGFzcyBYKFxuICAgICAgICB2YWwgJE5BTUU6ICRUWVBFXG4gICAgICApXG4gICAgc2VsZWN0b3I6IGNsYXNzX3BhcmFtZXRlclxuICBpbnNpZGU6XG4gICAgc3RvcEJ5OiBlbmRcbiAgICBwYXR0ZXJuOiB8XG4gICAgICBAU2VyaWFsaXphYmxlXG4gICAgICBkYXRhIGNsYXNzICQkJFxuIiwic291cmNlIjoicGFja2FnZSBjb20uZXhhbXBsZVxuXG5pbXBvcnQga290bGlueC5zZXJpYWxpemF0aW9uLlNlcmlhbGl6YWJsZVxuaW1wb3J0IGphdmEudXRpbC5VVUlEXG5cbkBTZXJpYWxpemFibGVcbmRhdGEgY2xhc3MgQW5ub3RhdGVkKFxuICAgIHZhbCBhOiBJbnQsXG4gICAgdmFsIGI6IFVVSUQsXG4pXG5cbmRhdGEgY2xhc3MgTm90QW5ub3RhdGVkKFxuICAgIHZhbCBhOiBJbnQsXG4gICAgdmFsIGI6IFVVSUQsXG4pXG4ifQ==)

As mentioned in the issue title, the Playground works as expected.


# 💻 Code

<!-- Please post the relevant code sample here as well.
  This code and the Playground code should be the same. -->
Code:

```kt
package com.example

import kotlinx.serialization.Serializable
import java.util.UUID

@Serializable
data class Annotated(
    val a: Int,
    val b: UUID,
)

data class NotAnnotated(
    val a: Int,
    val b: UUID,
)
```

Pattern:

```yml
id: parameter-in-serializable-class
language: kotlin
rule:
  pattern:
    context: |
      class X(
        val $NAME: $TYPE
      )
    selector: class_parameter
  inside:
    stopBy: end
    pattern: |
      @Serializable
      data class $$$
```

# 🙁 Actual behavior

<!-- What happened, and why it was wrong -->

`sg scan --rule parameter-in-serializable-class.yml Test.kt` does not match at all.

<img width="674" alt="image" src="https://github.com/ast-grep/ast-grep/assets/532251/847486f6-b7b9-45ac-a75a-54968eb43600">



Note that when I comment out the `inside` section of the rule, `sg scan --rule after-commen-out-inside.yml Test.kt` matches with the constructor parameters.

<img width="653" alt="image" src="https://github.com/ast-grep/ast-grep/assets/532251/4f2d9e81-0f80-4fa2-ad03-e8e4d287d950">

# 🙂 Expected behavior

<!-- What you expected to happen instead, and why -->

`sg scan --rule parameter-in-serializable-class.yml Test.kt` should match with constructor parameters inside a data class that is annotated with the `@Serializable`.

As mentioned in the issue title, the Playground works as expected.

<img width="1237" alt="image" src="https://github.com/ast-grep/ast-grep/assets/532251/ba0ef123-5c4b-43b1-9d92-d49693d9b18d">

# Environment

* ast-grep 0.23.0 (installed via homebrew)
* macOS Sonoma 14.5


Discussion / hints:
Hi! Thanks for the report! I don't have time to give it a look at the moment but I will give it back as soon as possible!
Thank you for the patience!
The issue is 

```
@Serializable data class $$$
```

is not parsed as class constructor in tree-sitter-kotlin 0.3.6

See the official parser page https://fwcd.dev/tree-sitter-kotlin/

<img width="499" alt="image" src="https://github.com/ast-grep/ast-grep/assets/2883231/b0e6c810-f887-48cf-b508-a1cd5228b86d">


However, ast-grep playground is using older kotlin parser so it somehow works. 


UPDATE: I have updated the tree-sitter-kotlin wasm on playground
In the case of tree-sitter parser cannot parse your pattern, using relational + kind rule is easier.
See [this example](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImtvdGxpbiIsInF1ZXJ5IjoiQFNlcmlhbGl6YWJsZVxuZGF0YSBjbGFzcyAkQSIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogcGFyYW1ldGVyLWluLXNlcmlhbGl6YWJsZS1jbGFzc1xubGFuZ3VhZ2U6IGtvdGxpblxucnVsZTpcbiAgcGF0dGVybjpcbiAgICBjb250ZXh0OiB8XG4gICAgICBjbGFzcyBYKFxuICAgICAgICB2YWwgJE5BTUU6ICRUWVBFXG4gICAgICApXG4gICAgc2VsZWN0b3I6IGNsYXNzX3BhcmFtZXRlclxuICBpbnNpZGU6XG4gICAga2luZDogcHJpbWFyeV9jb25zdHJ1Y3RvclxuICAgIGZvbGxvd3M6XG4gICAgICBwYXR0ZXJuOlxuICAgICAgICBjb250ZXh0OiAnQFNlcmlhbGl6YWJsZSBkYXRhIGNsYXNzIEEoYTogSW50KSdcbiAgICAgICAgc2VsZWN0b3I6IG1vZGlmaWVyc1xuICAgICAgc3RvcEJ5OiBlbmQiLCJzb3VyY2UiOiJAU2VyaWFsaXphYmxlXG5kYXRhIGNsYXNzIEFubm90YXRlZChcbiAgICB2YWwgYTogSW50LFxuICAgIHZhbCBiOiBVVUlELFxuKVxuIn0=)

<img width="929" alt="image" src="https://github.com/ast-grep/ast-grep/assets/2883231/47be27bd-7c29-4587-855a-db6d586e5821">

I also tested it on command line it works across different parser versions.
<img width="562" alt="image" src="https://github.com/ast-grep/ast-grep/assets/2883231/2b1d5b74-3d64-466a-a06e-543442cfe18b">

@HerringtonDarkholme Thank you for your prompt response! I was able to extract the class parameters as intended with the rules you suggested.

> The issue is
>
>     @Serializable data class $$$
>
> is not parsed as class constructor in tree-sitter-kotlin 0.3.6

However, I am really sorry, but I feel that this explanation does not accurately describe the cause. Because the CLI does not show any parsing errors for pattern, and it seems natural that tree-sitter Kotlin cannot parse code with meta variables as it is.

Upon further reflection after receiving the response, I realized that I could make the reproducible code much smaller. In the example below, the relational rule is no longer used.

Still, the situation remains the same: no match in the CLI, but a match in the playground. Is there any other possible cause?

------

# ⏯ Playground Link

[Playground link with relevant code](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImtvdGxpbiIsInF1ZXJ5IjoiY29uc29sZS5sb2coJE1BVENIKSIsInJld3JpdGUiOiJsb2dnZXIubG9nKCRNQVRDSCkiLCJjb25maWciOiJpZDogc2VyaWFsaXphYmxlLWNsYXNzXG5sYW5ndWFnZToga290bGluXG5ydWxlOlxuICBwYXR0ZXJuOiB8XG4gICAgQFNlcmlhbGl6YWJsZVxuICAgIGRhdGEgY2xhc3MgJCQkXG4iLCJzb3VyY2UiOiJwYWNrYWdlIGNvbS5leGFtcGxlXG5cbmltcG9ydCBrb3RsaW54LnNlcmlhbGl6YXRpb24uU2VyaWFsaXphYmxlXG5pbXBvcnQgamF2YS51dGlsLlVVSURcblxuQFNlcmlhbGl6YWJsZVxuZGF0YSBjbGFzcyBBbm5vdGF0ZWQoXG4gICAgdmFsIGE6IEludCxcbiAgICB2YWwgYjogVVVJRCxcbilcblxuZGF0YSBjbGFzcyBOb3RBbm5vdGF0ZWQoXG4gICAgdmFsIGE6IEludCxcbiAgICB2YWwgYjogVVVJRCxcbilcbiJ9)

# 💻 Code
Code (Unchanged from the original):

```kt
package com.example

import kotlinx.serialization.Serializable
import java.util.UUID

@Serializable
data class Annotated(
    val a: Int,
    val b: UUID,
)

data class NotAnnotated(
    val a: Int,
    val b: UUID,
)
```

Pattern:

```yml
id: serializable-class
language: kotlin
rule:
  pattern: |
    @Serializable
    data class $$$
```

# 🙁 Actual behavior

The CLI does not match.

<img width="546" alt="image" src="https://github.com/ast-grep/ast-grep/assets/532251/ae3910c4-5c01-4307-8ad5-8cd6e50521c2">

# 🙂 Expected behavior

The CLI should match with the annotated class as the playground does.

<img width="1247" alt="image" src="https://github.com/ast-grep/ast-grep/assets/532251/d5defb73-8258-4125-a554-6053b6ecd103">


I have updated the title of the issue. My main concern is the difference between CLI and playground. Because such a difference would make rule development more difficult.

If the difference is due to differences in library versions, it would be nice if either of the following could be accomplished.

- Both the CLI and the playground show the version of the ast-grep and depending libraries including the tree-sitter so that a user can notice the difference of the versions.
- The CLI provides something like a debug mode that shows AST structure of the pattern and the input code so that a user can notice the difference of the parsed structure.
Thanks 👍 Having a cli debug output will be very helpful. 
I will implement it in near future 
One more thing is to note is the behavior of treesitter. 
At most time web and native tree sitter behave the same. However, when the input is not valid, which is commonly seen in pattern, web and native will behave differently due to tree sitter heuristics. 

TS will try to recover from parsing error and the recovery is based on the input length. Sadly, web and native tree sitter have different string encoding so their length are different. 

This makes different parsing results 
That is a fascinating explanation for why tree sitter has different behavior in a browser vs natively.
Here's an example of tree sitter behaving differently for strings of different lengths:

![GIF of tree sitter behavior](https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExMmc5dm1jbHB1Y2U2ZDBkd214a2JxNXExbThxcGtsMTJybGhiejZvdCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/EvBSeL6mE1ySFHrKVr/giphy.gif)
@HerringtonDarkholme @StevenLove Thank you for clarifying the difference between web and native. I understand that the parsing result depends on the length of the input.

I am interested in the different encodings that affect the length of the input. Personally, if there is an option to use the same encoding in the CLI and playground, I would prefer to use that, even if it slows down the execution.  This is because I often refer to the playground results when developing ast-grep rules, and I consider the consistency of the results to be important.

Fix patch:
diff --git a/Cargo.lock b/Cargo.lock
index 7bcbada03c..a7d08cbb81 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -140,6 +140,7 @@ dependencies = [
  "similar",
  "tempfile",
  "tokio",
+ "tree-sitter-facade-sg",
 ]
 
 [[package]]
diff --git a/crates/cli/Cargo.toml b/crates/cli/Cargo.toml
index d0768556f6..ff809e5e42 100644
--- a/crates/cli/Cargo.toml
+++ b/crates/cli/Cargo.toml
@@ -31,6 +31,7 @@ ast-grep-config.workspace = true
 ast-grep-dynamic.workspace = true
 ast-grep-language.workspace = true
 ast-grep-lsp.workspace = true
+tree-sitter.workspace = true
 
 ansi_term = "0.12.1"
 anyhow.workspace = true
diff --git a/crates/cli/src/debug.rs b/crates/cli/src/debug.rs
new file mode 100644
index 0000000000..6001b95c4e
--- /dev/null
+++ b/crates/cli/src/debug.rs
@@ -0,0 +1,176 @@
+use crate::lang::SgLang;
+use crate::print::ColorArg;
+use ansi_term::Style;
+use ast_grep_core::Pattern;
+use ast_grep_language::Language;
+use clap::ValueEnum;
+use tree_sitter as ts;
+
+#[derive(Clone, Copy, PartialEq, Eq, ValueEnum)]
+pub enum DebugFormat {
+  Pattern,
+  Ast,
+  Cst,
+  Sexp,
+}
+impl DebugFormat {
+  pub fn debug_query(&self, pattern: &str, lang: SgLang, color: ColorArg) {
+    let colored = color.should_use_color();
+    match self {
+      DebugFormat::Pattern => {
+        let pattern = Pattern::try_new(pattern, lang).expect("pattern must be validated in run");
+        println!("Debug Pattern:\n{:?}", pattern);
+      }
+      DebugFormat::Sexp => {
+        let root = lang.ast_grep(pattern);
+        println!("Debug Sexp:\n{}", root.root().to_sexp());
+      }
+      DebugFormat::Ast => {
+        let root = lang.ast_grep(pattern);
+        let dumped = dump_node(root.root().get_ts_node());
+        println!("Debug AST:\n{}", dumped.ast(colored));
+      }
+      DebugFormat::Cst => {
+        let root = lang.ast_grep(pattern);
+        let dumped = dump_node(root.root().get_ts_node());
+        println!("Debug CST:\n{}", dumped.cst(colored));
+      }
+    }
+  }
+}
+
+pub struct DumpNode {
+  field: Option<String>,
+  kind: String,
+  start: Pos,
+  end: Pos,
+  is_named: bool,
+  children: Vec<DumpNode>,
+}
+
+struct DumpFmt {
+  kind_style: Style,
+  field_style: Style,
+  named_only: bool,
+}
+
+impl DumpFmt {
+  fn named(colored: bool) -> Self {
+    let style = Style::new();
+    Self {
+      kind_style: if colored { style.bold() } else { style },
+      field_style: if colored { style.italic() } else { style },
+      named_only: true,
+    }
+  }
+  fn all(colored: bool) -> Self {
+    let style = Style::new();
+    Self {
+      kind_style: if colored { style.bold() } else { style },
+      field_style: if colored { style.italic() } else { style },
+      named_only: false,
+    }
+  }
+}
+
+// TODO: add colorized output
+use std::fmt::{Result as FmtResult, Write};
+impl DumpNode {
+  pub fn ast(&self, colored: bool) -> String {
+    let mut result = String::new();
+    let fmt = DumpFmt::named(colored);
+    self
+      .helper(&mut result, &fmt, 0)
+      .expect("should write string");
+    result
+  }
+
+  pub fn cst(&self, colored: bool) -> String {
+    let mut result = String::new();
+    let fmt = DumpFmt::all(colored);
+    self
+      .helper(&mut result, &fmt, 0)
+      .expect("should write string");
+    result
+  }
+
+  fn helper(&self, result: &mut String, fmt: &DumpFmt, depth: usize) -> FmtResult {
+    let indent = "  ".repeat(depth);
+    if fmt.named_only && !self.is_named {
+      return Ok(());
+    }
+    write!(result, "{indent}")?;
+    if let Some(field) = &self.field {
+      let field = fmt.field_style.paint(field);
+      write!(result, "{}: ", field)?;
+    }
+    write!(result, "{}", fmt.kind_style.paint(&self.kind))?;
+    writeln!(result, " ({:?})-({:?})", self.start, self.end)?;
+    for child in &self.children {
+      child.helper(result, fmt, depth + 1)?;
+    }
+    Ok(())
+  }
+}
+
+pub struct Pos {
+  row: u32,
+  column: u32,
+}
+
+impl std::fmt::Debug for Pos {
+  fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+    write!(f, "{},{}", self.row, self.column)
+  }
+}
+
+impl From<ts::Point> for Pos {
+  #[inline]
+  fn from(pt: ts::Point) -> Self {
+    Pos {
+      row: pt.row(),
+      column: pt.column(),
+    }
+  }
+}
+
+fn dump_node(node: ts::Node) -> DumpNode {
+  let mut cursor = node.walk();
+  let mut nodes = vec![];
+  dump_one_node(&mut cursor, &mut nodes);
+  nodes.pop().expect("should have at least one node")
+}
+
+fn dump_one_node(cursor: &mut ts::TreeCursor, target: &mut Vec<DumpNode>) {
+  let node = cursor.node();
+  let kind = if node.is_missing() {
+    format!("MISSING {}", node.kind())
+  } else {
+    node.kind().to_string()
+  };
+  let start = node.start_position().into();
+  let end = node.end_position().into();
+  let field = cursor.field_name().map(|c| c.to_string());
+  let mut children = vec![];
+  if cursor.goto_first_child() {
+    dump_nodes(cursor, &mut children);
+    cursor.goto_parent();
+  }
+  target.push(DumpNode {
+    field,
+    kind,
+    start,
+    end,
+    children,
+    is_named: node.is_named(),
+  })
+}
+
+fn dump_nodes(cursor: &mut ts::TreeCursor, target: &mut Vec<DumpNode>) {
+  loop {
+    dump_one_node(cursor, target);
+    if !cursor.goto_next_sibling() {
+      break;
+    }
+  }
+}
diff --git a/crates/cli/src/lib.rs b/crates/cli/src/lib.rs
index fe701160ae..6a06cc5209 100644
--- a/crates/cli/src/lib.rs
+++ b/crates/cli/src/lib.rs
@@ -1,5 +1,6 @@
 mod completions;
 mod config;
+mod debug;
 mod error;
 mod lang;
 mod lsp;
@@ -166,6 +167,9 @@ mod test_cli {
     ok("run -p test --interactive dir");
     ok("run -p test -r Test dir");
     ok("run -p test -l rs --debug-query");
+    ok("run -p test -l rs --debug-query not");
+    ok("run -p test -l rs --debug-query=ast");
+    ok("run -p test -l rs --debug-query=cst");
     ok("run -p test -l rs --color always");
     ok("run -p test -l rs --heading always");
     ok("run -p test dir1 dir2 dir3"); // multiple paths
@@ -174,6 +178,8 @@ mod test_cli {
     ok("run -p test --json compact"); // argument after --json should not be parsed as JsonStyle
     ok("run -p test --json=pretty dir");
     ok("run -p test --json dir"); // arg after --json should not be parsed as JsonStyle
+    ok("run -p test --strictness ast");
+    ok("run -p test --strictness relaxed");
     error("run test");
     error("run --debug-query test"); // missing lang
     error("run -r Test dir");
@@ -181,6 +187,8 @@ mod test_cli {
     error("run -p test -l rs -c always"); // no color shortcut
     error("run -p test -U");
     error("run -p test --update-all");
+    error("run -p test --strictness not");
+    error("run -p test -l rs --debug-query=not");
   }
 
   #[test]
diff --git a/crates/cli/src/print/colored_print.rs b/crates/cli/src/print/colored_print.rs
index 22513b2211..48884c1c68 100644
--- a/crates/cli/src/print/colored_print.rs
+++ b/crates/cli/src/print/colored_print.rs
@@ -628,7 +628,7 @@ impl From<ColorChoice> for PrintStyles {
 }
 
 // copied from termcolor
-mod choose_color {
+pub(crate) mod choose_color {
   use super::ColorChoice;
   use std::env;
   /// Returns true if we should attempt to write colored output.
diff --git a/crates/cli/src/print/mod.rs b/crates/cli/src/print/mod.rs
index e01fbe7fe0..b86ccc7ceb 100644
--- a/crates/cli/src/print/mod.rs
+++ b/crates/cli/src/print/mod.rs
@@ -105,6 +105,13 @@ pub enum ColorArg {
   Never,
 }
 
+impl ColorArg {
+  pub fn should_use_color(self) -> bool {
+    use colored_print::choose_color::should_use_color;
+    should_use_color(&self.into())
+  }
+}
+
 impl From<ColorArg> for ColorChoice {
   fn from(arg: ColorArg) -> ColorChoice {
     use ColorArg::*;
diff --git a/crates/cli/src/run.rs b/crates/cli/src/run.rs
index e4b4934f74..b80ee4c6be 100644
--- a/crates/cli/src/run.rs
+++ b/crates/cli/src/run.rs
@@ -8,6 +8,7 @@ use clap::{builder::PossibleValue, Parser, ValueEnum};
 use ignore::WalkParallel;
 
 use crate::config::register_custom_language;
+use crate::debug::DebugFormat;
 use crate::error::ErrorContext as EC;
 use crate::lang::SgLang;
 use crate::print::{ColoredPrinter, Diff, Heading, InteractivePrinter, JSONPrinter, Printer};
@@ -54,7 +55,7 @@ impl ValueEnum for Strictness {
       M::Cst => PossibleValue::new("cst").help("Match exact all node"),
       M::Smart => PossibleValue::new("smart").help("Match all node except source trivial nodes"),
       M::Ast => PossibleValue::new("ast").help("Match only ast nodes"),
-      M::Relaxed => PossibleValue::new("lenient").help("Match ast node except comments"),
+      M::Relaxed => PossibleValue::new("relaxed").help("Match ast node except comments"),
       M::Signature => {
         PossibleValue::new("signature").help("Match ast node except comments, without text")
       }
@@ -78,8 +79,15 @@ pub struct RunArg {
   lang: Option<SgLang>,
 
   /// Print query pattern's tree-sitter AST. Requires lang be set explicitly.
-  #[clap(long, requires = "lang")]
-  debug_query: bool,
+  #[clap(
+      long,
+      requires = "lang",
+      value_name="format",
+      num_args(0..=1),
+      require_equals = true,
+      default_missing_value = "pattern"
+  )]
+  debug_query: Option<DebugFormat>,
 
   /// The strictness of the pattern.
   #[clap(long)]
@@ -251,8 +259,8 @@ impl<P: Printer> Worker for RunWithSpecificLang<P> {
     printer.before_print()?;
     let arg = &self.arg;
     let lang = arg.lang.expect("must present");
-    if arg.debug_query {
-      println!("Pattern TreeSitter {:?}", self.pattern);
+    if let Some(format) = arg.debug_query {
+      format.debug_query(&self.arg.pattern, lang, self.arg.output.color);
     }
     let rewrite = if let Some(s) = &arg.rewrite {
       Some(Fixer::from_str(s, &lang).context(EC::ParsePattern)?)
@@ -331,7 +339,7 @@ mod test {
       rewrite: None,
       lang: None,
       heading: Heading::Never,
-      debug_query: false,
+      debug_query: None,
       strictness: None,
       input: InputArgs {
         no_ignore: vec![],


Test patch: