Repository: goccy/go-yaml
Pull request #521 (2024-11-11 15:06:01)

Fix parsing of escaped new-line-character or white-space in double quote text
fix https://github.com/goccy/go-yaml/issues/344

codecov-commenter: :warning: Please install the !['codecov app svg image'](https://github.com/codecov/engineering-team/assets/152432831/e90313f4-9d3a-4b63-8b54-cfe14e7ec20d) to ensure uploads and comments are reliably processed by Codecov.

## [Codecov](https://app.codecov.io/gh/goccy/go-yaml/pull/521?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Masaaki+Goshima) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 79.16%. Comparing base [(`7e833a2`)](https://app.codecov.io/gh/goccy/go-yaml/commit/7e833a2d40134729e82c98cd5862ded2bb91e1ad?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Masaaki+Goshima) to head [(`9e58f7e`)](https://app.codecov.io/gh/goccy/go-yaml/commit/9e58f7eeda83f7c5aede71e28205863b61476ff0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Masaaki+Goshima).

:exclamation: Your organization needs to install the [Codecov GitHub app](https://github.com/apps/codecov/installations/select_target) to enable full functionality.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master     #521      +/-   ##
==========================================
+ Coverage   79.10%   79.16%   +0.05%     
==========================================
  Files          17       17              
  Lines        6112     6129      +17     
==========================================
+ Hits         4835     4852      +17     
  Misses        965      965              
  Partials      312      312              
```

</details>

Patch:
diff --git a/scanner/scanner.go b/scanner/scanner.go
index 746b8c7..8739bf5 100644
--- a/scanner/scanner.go
+++ b/scanner/scanner.go
@@ -294,7 +294,15 @@ func (s *Scanner) scanDoubleQuote(ctx *Context) (*token.Token, error) {
 		c := src[idx]
 		ctx.addOriginBuf(c)
 		if s.isNewLineChar(c) {
-			value = append(value, ' ')
+			if isFirstLineChar {
+				if value[len(value)-1] == ' ' {
+					value[len(value)-1] = '\n'
+				} else {
+					value = append(value, '\n')
+				}
+			} else {
+				value = append(value, ' ')
+			}
 			isFirstLineChar = true
 			isNewLine = true
 			s.progressLine(ctx)
@@ -388,6 +396,16 @@ func (s *Scanner) scanDoubleQuote(ctx *Context) (*token.Token, error) {
 				progress = 1
 				ctx.addOriginBuf(nextChar)
 				value = append(value, c)
+			case '\n':
+				isFirstLineChar = true
+				isNewLine = true
+				ctx.addOriginBuf(nextChar)
+				s.progressColumn(ctx, 1)
+				s.progressLine(ctx)
+				idx++
+				continue
+			case ' ':
+				// skip escape character.
 			default:
 				value = append(value, c)
 			}


Test patch:
diff --git a/lexer/lexer_test.go b/lexer/lexer_test.go
index 9cae3f1..b57fff4 100644
--- a/lexer/lexer_test.go
+++ b/lexer/lexer_test.go
@@ -550,6 +550,41 @@ func TestTokenize(t *testing.T) {
 				},
 			},
 		},
+		{
+			YAML: `
+a:
+  "bbb  \
+      ccc
+
+      ddd eee\n\
+  \ \ fff ggg\nhhh iii\n
+  jjj kkk
+  "
+`,
+			Tokens: token.Tokens{
+				{
+					Type:          token.StringType,
+					CharacterType: token.CharacterTypeMiscellaneous,
+					Indicator:     token.NotIndicator,
+					Value:         "a",
+					Origin:        "\na",
+				},
+				{
+					Type:          token.MappingValueType,
+					CharacterType: token.CharacterTypeIndicator,
+					Indicator:     token.BlockStructureIndicator,
+					Value:         ":",
+					Origin:        ":",
+				},
+				{
+					Type:          token.DoubleQuoteType,
+					CharacterType: token.CharacterTypeIndicator,
+					Indicator:     token.QuotedScalarIndicator,
+					Value:         "bbb  ccc\nddd eee\n  fff ggg\nhhh iii\n jjj kkk ",
+					Origin:        "\n  \"bbb  \\\n      ccc\n\n      ddd eee\\n\\\n  \\ \\ fff ggg\\nhhh iii\\n\n  jjj kkk\n  \"",
+				},
+			},
+		},
 		{
 			YAML: `v: null
 		`,
@@ -2948,7 +2983,7 @@ foo2: 'bar2'`,
 				{
 					line:   1,
 					column: 6,
-					value:  "test     bar",
+					value:  "test\n\n\n\nbar",
 				},
 				{
 					line:   7,