Repository: ruby/prism (c)

Issue:
`Prism::Translation::Parser` incompatibility AST for string literal with line breaks
> [!NOTE]
> This might be the same as #2506, but I'm reporting it as a different issue due to a slightly different context. Please close it if it can be treated the same as #2506.

Incompatibility AST for string literal with line breaks between Parser gem and Prism:

```ruby
$ cat example.rb
"foo
  bar"
```

## Expected Behavior

Returns a `dsym` node:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse_file("example.rb")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:dstr,
  s(:str, "foo\n"),
  s(:str, "  bar"))
```

## Actual Behavior

Returns a `str` node:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse_file("example.rb")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:str, "foo\n  bar")
```

## Background

Found due to incompatibility with RuboCop's `Style/StringLiterals` cop.

Pull request description:
Fix an AST and token incompatibility for `Prism::Translation::Parser`
Fixes #2515.

This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser` for string literal with line breaks.

Patch:
diff --git a/lib/prism/translation/parser/compiler.rb b/lib/prism/translation/parser/compiler.rb
index d64e382c..bf5b8544 100644
--- a/lib/prism/translation/parser/compiler.rb
+++ b/lib/prism/translation/parser/compiler.rb
@@ -1487,9 +1487,23 @@ module Prism
           elsif node.opening == "?"
             builder.character([node.unescaped, srange(node.location)])
           else
+            parts = if node.unescaped.lines.count <= 1
+              [builder.string_internal([node.unescaped, srange(node.content_loc)])]
+            else
+              start_offset = node.content_loc.start_offset
+
+              node.unescaped.lines.map do |line|
+                end_offset = start_offset + line.length
+                offsets = srange_offsets(start_offset, end_offset)
+                start_offset = end_offset
+
+                builder.string_internal([line, offsets])
+              end
+            end
+
             builder.string_compose(
               token(node.opening_loc),
-              [builder.string_internal([node.unescaped, srange(node.content_loc)])],
+              parts,
               token(node.closing_loc)
             )
           end
diff --git a/lib/prism/translation/parser/lexer.rb b/lib/prism/translation/parser/lexer.rb
index b28273b0..8bb30845 100644
--- a/lib/prism/translation/parser/lexer.rb
+++ b/lib/prism/translation/parser/lexer.rb
@@ -281,7 +281,7 @@ module Prism
                 value = ""
                 location = Range.new(source_buffer, offset_cache[next_location.start_offset], offset_cache[next_location.end_offset])
                 index += 1
-              elsif ["\"", "'"].include?(value) && (next_token = lexed[index][0]) && next_token.type == :STRING_CONTENT && (next_next_token = lexed[index + 1][0]) && next_next_token.type == :STRING_END
+              elsif ["\"", "'"].include?(value) && (next_token = lexed[index][0]) && next_token.type == :STRING_CONTENT && next_token.value.lines.count <= 1 && (next_next_token = lexed[index + 1][0]) && next_next_token.type == :STRING_END
                 next_location = token.location.join(next_next_token.location)
                 type = :tSTRING
                 value = next_token.value


Test patch:
diff --git a/test/prism/fixtures/dstring.txt b/test/prism/fixtures/dstring.txt
new file mode 100644
index 00000000..b7a0958d
--- /dev/null
+++ b/test/prism/fixtures/dstring.txt
@@ -0,0 +1,2 @@
+"foo
+  bar"
diff --git a/test/prism/snapshots/dstring.txt b/test/prism/snapshots/dstring.txt
new file mode 100644
index 00000000..a24eaf19
--- /dev/null
+++ b/test/prism/snapshots/dstring.txt
@@ -0,0 +1,11 @@
+@ ProgramNode (location: (1,0)-(2,6))
+├── locals: []
+└── statements:
+    @ StatementsNode (location: (1,0)-(2,6))
+    └── body: (length: 1)
+        └── @ StringNode (location: (1,0)-(2,6))
+            ├── flags: ∅
+            ├── opening_loc: (1,0)-(1,1) = "\""
+            ├── content_loc: (1,1)-(2,5) = "foo\n  bar"
+            ├── closing_loc: (2,5)-(2,6) = "\""
+            └── unescaped: "foo\n  bar"