Repository: shikijs/shiki
Pull request #897 (2025-01-20 05:34:53)

feat(engine-js): improve performance for some languages
- Updates `oniguruma-to-es` to v2.2.0.
  - Reduces Razor errors from 5 to 3 via new support for group names that are invalid JS identifiers but valid in Oniguruma.
- Applies new option `rules.singleline` (Oniguruma option `ONIG_OPTION_SINGLELINE`).
  - Changes transpiled output for `^` and `$`.
  - Improves performance of some grammars without any change in meaning since TM grammars are applied line by line.
- Tweaks new perf-related documentation.

netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *shiki-matsu* processing.


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | c11e8a69af7f0264f1498a9bd4b4baa21545a004 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/shiki-matsu/deploys/678de07ff15bbf0008452ca4 |
netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *shiki-next* processing.


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | c11e8a69af7f0264f1498a9bd4b4baa21545a004 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/shiki-next/deploys/678de07fcb67760008f56db6 |

Patch:
diff --git a/docs/guide/best-performance.md b/docs/guide/best-performance.md
index 74c545f0..9e9652b9 100644
--- a/docs/guide/best-performance.md
+++ b/docs/guide/best-performance.md
@@ -4,7 +4,7 @@ outline: deep
 
 # Best Performance Practices
 
-This guide will help you to improve the performance your Shiki usage.
+This guide will help you to improve the performance of your Shiki usage.
 
 ## Cache the Highlighter Instance
 
@@ -23,7 +23,7 @@ export async function highlightCode(code: string, lang: string) {
 }
 ```
 
-When you no longer need a highlighter instance, you can call `dispose()` method to release the resources. (It can't not be GC-ed automatically, you need to do it explicitly)
+When you no longer need a highlighter instance, you can call the `dispose()` method to release the resources. (It can't be GC-ed automatically, you need to do it explicitly.)
 
 ```ts
 highlighter.dispose()
@@ -31,11 +31,11 @@ highlighter.dispose()
 
 ## Fine-Grained Bundle
 
-The pre-built bundles are for easy usage, and mostly intended for Node.js environment where you don't worry too much about the bundle size. If you are building a web application, or in a resource-constrained environment, it's always better to use the fine-grained bundles to reduce the bundle size and memory usage.
+The pre-built bundles are for easy usage, and mostly intended for a Node.js environment where you aren't worried about the bundle size. If you are building a web application or in a resource-constrained environment, it's always better to use the fine-grained bundles to reduce the bundle size and memory usage.
 
 **Avoid importing `shiki`, `shiki/bundle/full`, `shiki/bundle/web` directly**.
 
-Instead, import the fine-grained modules like `shiki/core`, `shiki/engine/javascript`, `@shikijs/langs/typescript`, `@shikijs/themes/dark-plus`, etc.
+Instead, import fine-grained modules like `shiki/core`, `shiki/engine/javascript`, `@shikijs/langs/typescript`, `@shikijs/themes/dark-plus`, etc.
 
 ```ts
 import { createHighlighterCore } from 'shiki/core'
@@ -56,13 +56,13 @@ const highlighter = await createHighlighterCore({
 })
 ```
 
-To compose the fine-grained bundles easily, we also provide the [`shiki-codegen`](/packages/codegen) tool to generate the fine-grained bundles for you.
+To compose the fine-grained bundles easily, we also provide the [`shiki-codegen`](/packages/codegen) tool to generate fine-grained bundles for you.
 
 Learn more about [Fine-Grained Bundles](/guide/bundles#fine-grained-bundle).
 
 ## Use Shorthands
 
-`createHighlighter` and `createHighlighterCore` loads all the themes and languages **upfront** to ensure the subsequent highlight operations are synchronous. This can add a significant overhead to the startup time specially when you have a lot of themes and languages. Shorthands abstract the theme and language loading process and maintains an internal highlighter instance underneath, and only load the necessary themes and languages when needed. When your highlight process can be asynchronous, you can use the shorthands to reduce the startup time.
+`createHighlighter` and `createHighlighterCore` load all the themes and languages **upfront** to ensure subsequent highlight operations are synchronous. This can add significant overhead to startup time, especially when you have a lot of themes and languages. Shorthands abstract the theme and language loading process and maintain an internal highlighter instance underneath, only loading the necessary themes and languages when needed. When your highlighting process can be asynchronous, you can use shorthands to reduce startup time.
 
 ```ts
 import { codeToHtml } from 'shiki'
@@ -78,15 +78,15 @@ You can also create your own shorthands with fine-grained bundles. Check out the
 
 ## JavaScript Engine and Pre-compiled Languages
 
-Shiki provides [two engines](/guide/regex-engines) for executing Regular Expressions: [`JavaScript`](/guide/regex-engines#javascript-regexp-engine) and [`Oniguruma`](/guide/regex-engines#oniguruma-engine). The Oniguruma engine is a WebAssembly-based that compiled from C++ code, and `JavaScript` is a pure JavaScript engine that translates Oniguruma-flavored regex to JavaScript regex.
+Shiki provides [two engines](/guide/regex-engines) for executing regular expressions: [`JavaScript`](/guide/regex-engines#javascript-regexp-engine) and [`Oniguruma`](/guide/regex-engines#oniguruma-engine). The Oniguruma engine is WebAssembly-based and compiled from C code, and `JavaScript` is a pure JavaScript engine that translates Oniguruma-flavored regexes to JavaScript regexes.
 
-If you are bundling Shiki for the web, using the JavaScript engine would be smaller in bundle size and faster in startup time. Meanwhile, the [precompiled languages](/guide/regex-engines#pre-compiled-languages) can also reduce the bundle size and startup time, if your target browsers support the latest RegExp features.
+If you are bundling Shiki for the web, using the JavaScript engine results in a smaller bundle size and faster startup time. The [precompiled languages](/guide/regex-engines#pre-compiled-languages) can further reduce bundle size and startup time, if your target browsers support the latest RegExp features.
 
 Check the [RegExp Engines](/guide/regex-engines) guide for more details.
 
 ## Use Workers
 
-Shiki hightlights the code using Regular Expressions, which can be CPU-intensive. You can offload the highlighting work to a Web Worker/Node Worker to avoid blocking the main thread.
+Shiki hightlights code using regular expressions, which can be CPU-intensive. You can offload the highlighting work to a Web Worker/Node Worker to avoid blocking the main thread.
 
 ::: info
 
diff --git a/docs/guide/regex-engines.md b/docs/guide/regex-engines.md
index 2fd17b00..188e4684 100644
--- a/docs/guide/regex-engines.md
+++ b/docs/guide/regex-engines.md
@@ -85,13 +85,13 @@ const jsEngine = createJavaScriptRegexEngine({
 
 ### Pre-compiled Languages
 
-Instead of compiling the regular expressions at on-the-fly, we also provided pre-compiled languages for the JavaScript engine to further reduce the startup time.
+Instead of compiling regular expressions on-the-fly, we also provide pre-compiled languages for the JavaScript engine to further reduce startup time.
 
 ::: info
-Pre-compiled languages requires RegExp uncoide sets support (`v` flag), which is targeting **ES2024** or Node.js 20+, and may not work in older environments. [Can I use](https://caniuse.com/mdn-javascript_builtins_regexp_unicode).
+Pre-compiled languages require support for RegExp UnicodeSets (the `v` flag), which requires **ES2024** or Node.js 20+, and may not work in older environments. [Can I use](https://caniuse.com/mdn-javascript_builtins_regexp_unicodesets).
 :::
 
-You can install them with `@shikijs/langs-precompiled`, and change your `@shikijs/langs` imports to `@shikijs/langs-precompiled`:
+You can install them with `@shikijs/langs-precompiled`, and then change your `@shikijs/langs` imports to `@shikijs/langs-precompiled`:
 
 ```ts
 import { createHighlighterCore } from 'shiki/core'
@@ -113,6 +113,6 @@ const highlighter = await createHighlighterCore({
 })
 ```
 
-If you are not using the custom grammars that requires transpilation, you can use the `createJavaScriptRawEngine` to skip the transpilation step further reducing bundle size.
+If you are not using custom grammars that require transpilation, you can use `createJavaScriptRawEngine` to skip the transpilation step, further reducing bundle size.
 
-If you are using [`shiki-codegen`](/packages/codegen), you can generate the pre-compiled languages with the `--precompiled` and `--engine=javascript-raw` flags.
+If you are using [`shiki-codegen`](/packages/codegen), you can generate pre-compiled languages with the `--precompiled` and `--engine=javascript-raw` flags.
diff --git a/docs/references/engine-js-compat.md b/docs/references/engine-js-compat.md
index 85c19bcb..af895568 100644
--- a/docs/references/engine-js-compat.md
+++ b/docs/references/engine-js-compat.md
@@ -2,9 +2,9 @@
 
 Compatibility reference of all built-in grammars with the [JavaScript RegExp engine](/guide/regex-engines#javascript-regexp-engine).
 
-> Generated on Tuesday, January 14, 2025
+> Generated on Monday, January 20, 2025
 >
-> Version `1.27.0`
+> Version `1.27.2`
 >
 > Runtime: Node.js v22.11.0
 
@@ -262,5 +262,5 @@ Languages that throw with the JavaScript RegExp engine, either because they cont
 | sass       | ✅ OK           |                67 |               2 |      |
 | purescript | ❌ Error        |                72 |               1 |      |
 | csharp     | ❌ Error        |               310 |               3 |  137 |
+| razor      | ❌ Error        |               959 |               3 |      |
 | swift      | ❌ Error        |               326 |               3 |      |
-| razor      | ❌ Error        |               957 |               5 |      |
diff --git a/packages/engine-javascript/src/engine-compile.ts b/packages/engine-javascript/src/engine-compile.ts
index 5f46c9cd..4346e237 100644
--- a/packages/engine-javascript/src/engine-compile.ts
+++ b/packages/engine-javascript/src/engine-compile.ts
@@ -41,6 +41,9 @@ export function defaultJavaScriptRegexConstructor(pattern: string, options?: Oni
         // Oniguruma uses depth limit `20`; lowered here to keep regexes shorter and maybe
         // sometimes faster, but can be increased if issues reported due to low limit
         recursionLimit: 5,
+        // Oniguruma option for `^`->`\A`, `$`->`\Z`; improves search performance without any
+        // change in meaning since TM grammars search line by line
+        singleline: true,
       },
       ...options,
     },
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index d1088c9e..290e53e4 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -148,8 +148,8 @@ catalogs:
       specifier: ^1.1.4
       version: 1.1.4
     oniguruma-to-es:
-      specifier: ^2.0.0
-      version: 2.0.0
+      specifier: ^2.2.0
+      version: 2.2.0
     picocolors:
       specifier: ^1.1.1
       version: 1.1.1
@@ -578,7 +578,7 @@ importers:
         version: 10.0.1
       oniguruma-to-es:
         specifier: 'catalog:'
-        version: 2.0.0
+        version: 2.2.0
 
   packages/engine-oniguruma:
     dependencies:
@@ -610,7 +610,7 @@ importers:
         version: link:../types
       oniguruma-to-es:
         specifier: 'catalog:'
-        version: 2.0.0
+        version: 2.2.0
     devDependencies:
       tm-grammars:
         specifier: 'catalog:'
@@ -4116,8 +4116,8 @@ packages:
     resolution: {integrity: sha512-VXJjc87FScF88uafS3JllDgvAm+c/Slfz06lorj2uAY34rlUu0Nt+v8wreiImcrgAjjIHp1rXpTDlLOGw29WwQ==}
     engines: {node: '>=18'}
 
-  oniguruma-to-es@2.0.0:
-    resolution: {integrity: sha512-pE7+9jQgomy10aK6BJKRNHj1Nth0YLOzb3iRuhlz4gRzNSBSd7hga6U8BE6o0SoSuSkqv+PPtt511Msd1Hkl0w==}
+  oniguruma-to-es@2.2.0:
+    resolution: {integrity: sha512-EEsso27ri0sf+t4uRFEj5C5gvXQj0d0w1Y2qq06b+hDLBnvzO1rWTwEW4C7ytan6nhg4WPwE26eLoiPhHUbvKg==}
 
   optionator@0.9.4:
     resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==}
@@ -9142,7 +9142,7 @@ snapshots:
     dependencies:
       mimic-function: 5.0.1
 
-  oniguruma-to-es@2.0.0:
+  oniguruma-to-es@2.2.0:
     dependencies:
       emoji-regex-xs: 1.0.0
       regex: 5.1.1
diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml
index f592debf..65829003 100644
--- a/pnpm-workspace.yaml
+++ b/pnpm-workspace.yaml
@@ -56,7 +56,7 @@ catalog:
   monaco-editor-core: ^0.52.2
   ofetch: ^1.4.1
   ohash: ^1.1.4
-  oniguruma-to-es: ^2.0.0
+  oniguruma-to-es: ^2.2.0
   picocolors: ^1.1.1
   pinia: ^2.3.0
   pnpm: ^9.15.4


Test patch:
diff --git a/packages/langs-precompiled/scripts/__snapshots__/precompile.test.ts.snap b/packages/langs-precompiled/scripts/__snapshots__/precompile.test.ts.snap
index 91bea6db..aa363561 100644
--- a/packages/langs-precompiled/scripts/__snapshots__/precompile.test.ts.snap
+++ b/packages/langs-precompiled/scripts/__snapshots__/precompile.test.ts.snap
@@ -13,14 +13,14 @@ exports[`precompile 1`] = `
     "syntax",
     "sublime-syntax",
   ],
-  firstLineMatch: /(?<=^|\\n(?!$))%YAML( ?1[^\\n]\\p{Nd}+)?/dgv,
+  firstLineMatch: /^%YAML( ?1[^\\n]\\p{Nd}+)?/dgv,
   name: "yaml",
   patterns: [
     { include: "#comment" },
     { include: "#property" },
     { include: "#directive" },
-    { match: /(?<=^|\\n(?!$))---/dgv, name: "entity.other.document.begin.yaml" },
-    { match: /(?<=^|\\n(?!$))\\.{3}/dgv, name: "entity.other.document.end.yaml" },
+    { match: /^---/dgv, name: "entity.other.document.begin.yaml" },
+    { match: /^\\.{3}/dgv, name: "entity.other.document.end.yaml" },
     { include: "#node" },
   ],
   repository: {
@@ -44,7 +44,7 @@ exports[`precompile 1`] = `
           beginCaptures: {
             "1": { name: "punctuation.definition.key-value.begin.yaml" },
           },
-          end: /(?=\\?)|(?<=^|\\n(?!$)) *(:)|(:)/dgv,
+          end: /(?=\\?)|^ *(:)|(:)/dgv,
           endCaptures: {
             "1": { name: "punctuation.separator.key-value.mapping.yaml" },
             "2": { name: "invalid.illegal.expected-newline.yaml" },
@@ -54,8 +54,8 @@ exports[`precompile 1`] = `
         },
         {
           begin:
-            /(?=(?:[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-]\\P{space})([^\\p{space}\\:]|:\\P{space}|\\p{space}+(?![\\#\\p{space}]))*\\p{space}*:(\\p{space}|(?=$|\\n)))/dgv,
-          end: /(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n)))/dgv,
+            /(?=(?:[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-]\\P{space})([^\\p{space}\\:]|:\\P{space}|\\p{space}+(?![\\#\\p{space}]))*\\p{space}*:(\\p{space}|(?=\\n?$)))/dgv,
+          end: /(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$)))/dgv,
           patterns: [
             { include: "#flow-scalar-plain-out-implicit-type" },
             {
@@ -63,13 +63,13 @@ exports[`precompile 1`] = `
                 /[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-]\\P{space}/dgv,
               beginCaptures: { "0": { name: "entity.name.tag.yaml" } },
               contentName: "entity.name.tag.yaml",
-              end: /(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n)))/dgv,
+              end: /(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$)))/dgv,
               name: "string.unquoted.plain.out.yaml",
             },
           ],
         },
         {
-          match: /:(?=\\p{space}|(?=$|\\n))/dgv,
+          match: /:(?=\\p{space}|(?=\\n?$))/dgv,
           name: "punctuation.separator.key-value.mapping.yaml",
         },
       ],
@@ -91,15 +91,13 @@ exports[`precompile 1`] = `
           ],
         },
       },
-      end: /*@__PURE__*/ new EmulatedRegExp(
-        "(?<=^|\\\\n(?!$))(?=\\\\P{space})|(?!^)",
-        "dgv",
-        { strategy: "search_start_clip" },
-      ),
+      end: /*@__PURE__*/ new EmulatedRegExp("^(?=\\\\P{space})|(?!^)", "dgv", {
+        strategy: "search_start_clip",
+      }),
       patterns: [
         {
-          begin: /(?<=^|\\n(?!$))([ ]+)(?! )/dgv,
-          end: /(?<=^|\\n(?!$))(?!\\1|\\p{space}*(?=$|\\n))()/dgv,
+          begin: /^([ ]+)(?! )/dgv,
+          end: /^(?!\\1|\\p{space}*(?=\\n?$))()/dgv,
           name: "string.unquoted.block.yaml",
         },
       ],
@@ -110,7 +108,7 @@ exports[`precompile 1`] = `
     },
     comment: {
       begin:
-        /(?:((?<=^|\\n(?!$))[ \\t]*)|[ \\t]+)(?=#[[\\P{space}&&\\P{cntrl}&&\\P{Cn}&&\\P{Cs}]\\p{Zs}]*(?=$|\\n))/dgv,
+        /(?:(^[ \\t]*)|[ \\t]+)(?=#[[\\P{space}&&\\P{cntrl}&&\\P{Cn}&&\\P{Cs}]\\p{Zs}]*(?=\\n?$))/dgv,
       beginCaptures: {
         "1": { name: "punctuation.whitespace.comment.leading.yaml" },
       },
@@ -129,11 +127,11 @@ exports[`precompile 1`] = `
       ],
     },
     directive: {
-      begin: /(?<=^|\\n(?!$))%/dgv,
+      begin: /^%/dgv,
       beginCaptures: {
         "0": { name: "punctuation.definition.directive.begin.yaml" },
       },
-      end: /(?=(?=$|\\n)|[ \\t]+((?=$|\\n)|#))/dgv,
+      end: /(?=(?=\\n?$)|[ \\t]+((?=\\n?$)|#))/dgv,
       name: "meta.directive.yaml",
       patterns: [
         {
@@ -213,7 +211,7 @@ exports[`precompile 1`] = `
             { include: "#flow-pair" },
             { include: "#flow-node" },
             {
-              begin: /:(?=\\p{space}|(?=$|\\n)|[\\[\\]\\{\\}\\,])/dgv,
+              begin: /:(?=\\p{space}|(?=\\n?$)|[\\[\\]\\{\\}\\,])/dgv,
               beginCaptures: {
                 "0": { name: "punctuation.separator.key-value.mapping.yaml" },
               },
@@ -224,8 +222,8 @@ exports[`precompile 1`] = `
         },
         {
           begin:
-            /(?=(?:[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-][^\\p{space}\\[\\]\\{\\}\\,])([^\\p{space}\\:\\[\\]\\{\\}\\,]|:[^\\p{space}\\[\\]\\{\\}\\,]|\\p{space}+(?![\\#\\p{space}]))*\\p{space}*:(\\p{space}|(?=$|\\n)))/dgv,
-          end: /(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
+            /(?=(?:[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-][^\\p{space}\\[\\]\\{\\}\\,])([^\\p{space}\\:\\[\\]\\{\\}\\,]|:[^\\p{space}\\[\\]\\{\\}\\,]|\\p{space}+(?![\\#\\p{space}]))*\\p{space}*:(\\p{space}|(?=\\n?$)))/dgv,
+          end: /(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
           name: "meta.flow-pair.key.yaml",
           patterns: [
             { include: "#flow-scalar-plain-in-implicit-type" },
@@ -234,14 +232,14 @@ exports[`precompile 1`] = `
                 /[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-][^\\p{space}\\[\\]\\{\\}\\,]/dgv,
               beginCaptures: { "0": { name: "entity.name.tag.yaml" } },
               contentName: "entity.name.tag.yaml",
-              end: /(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
+              end: /(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
               name: "string.unquoted.plain.in.yaml",
             },
           ],
         },
         { include: "#flow-node" },
         {
-          begin: /:(?=\\p{space}|(?=$|\\n)|[\\[\\]\\{\\}\\,])/dgv,
+          begin: /:(?=\\p{space}|(?=\\n?$)|[\\[\\]\\{\\}\\,])/dgv,
           captures: {
             "0": { name: "punctuation.separator.key-value.mapping.yaml" },
           },
@@ -284,7 +282,7 @@ exports[`precompile 1`] = `
         {
           begin:
             /[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-][^\\p{space}\\[\\]\\{\\}\\,]/dgv,
-          end: /(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
+          end: /(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
           name: "string.unquoted.plain.in.yaml",
         },
       ],
@@ -302,7 +300,7 @@ exports[`precompile 1`] = `
             "7": { name: "constant.language.merge.yaml" },
           },
           match:
-            /(?:(null|Null|NULL|~)|(y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF)|((?:[\\-\\+]?0b[0-1_]+|[\\-\\+]?0[0-7_]+|[\\-\\+]?(?:0|[1-9][0-9_]*)|[\\-\\+]?0x[0-9a-fA-F_]+|[\\-\\+]?[1-9][0-9_]*(?::[0-5]?[0-9])+))|((?:[\\-\\+]?(?:[0-9][0-9_]*)?\\.[0-9\\.]*(?:[eE][\\-\\+][0-9]+)?|[\\-\\+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*|[\\-\\+]?\\.(?:inf|Inf|INF)|\\.(?:nan|NaN|NAN)))|((?:\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}|\\p{Nd}{4}-\\p{Nd}{1,2}-\\p{Nd}{1,2}(?:[Tt]|[ \\t]+)\\p{Nd}{1,2}:\\p{Nd}{2}:\\p{Nd}{2}(?:\\.\\p{Nd}*)?(?:[ \\t]*Z|[\\-\\+]\\p{Nd}{1,2}(?::\\p{Nd}{1,2})?)?))|(=)|(<<))(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
+            /(?:(null|Null|NULL|~)|(y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF)|((?:[\\-\\+]?0b[0-1_]+|[\\-\\+]?0[0-7_]+|[\\-\\+]?(?:0|[1-9][0-9_]*)|[\\-\\+]?0x[0-9a-fA-F_]+|[\\-\\+]?[1-9][0-9_]*(?::[0-5]?[0-9])+))|((?:[\\-\\+]?(?:[0-9][0-9_]*)?\\.[0-9\\.]*(?:[eE][\\-\\+][0-9]+)?|[\\-\\+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*|[\\-\\+]?\\.(?:inf|Inf|INF)|\\.(?:nan|NaN|NAN)))|((?:\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}|\\p{Nd}{4}-\\p{Nd}{1,2}-\\p{Nd}{1,2}(?:[Tt]|[ \\t]+)\\p{Nd}{1,2}:\\p{Nd}{2}:\\p{Nd}{2}(?:\\.\\p{Nd}*)?(?:[ \\t]*Z|[\\-\\+]\\p{Nd}{1,2}(?::\\p{Nd}{1,2})?)?))|(=)|(<<))(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$))|\\p{space}*:[\\[\\]\\{\\}\\,]|\\p{space}*[\\[\\]\\{\\}\\,])/dgv,
         },
       ],
     },
@@ -312,7 +310,7 @@ exports[`precompile 1`] = `
         {
           begin:
             /[^\\p{space}\\-\\?\\:\\,\\[\\]\\{\\}\\#\\&\\*\\!\\|\\>'"\\%\\@\\\`]|[\\?\\:\\-]\\P{space}/dgv,
-          end: /(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n)))/dgv,
+          end: /(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$)))/dgv,
           name: "string.unquoted.plain.out.yaml",
         },
       ],
@@ -330,7 +328,7 @@ exports[`precompile 1`] = `
             "7": { name: "constant.language.merge.yaml" },
           },
           match:
-            /(?:(null|Null|NULL|~)|(y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF)|((?:[\\-\\+]?0b[0-1_]+|[\\-\\+]?0[0-7_]+|[\\-\\+]?(?:0|[1-9][0-9_]*)|[\\-\\+]?0x[0-9a-fA-F_]+|[\\-\\+]?[1-9][0-9_]*(?::[0-5]?[0-9])+))|((?:[\\-\\+]?(?:[0-9][0-9_]*)?\\.[0-9\\.]*(?:[eE][\\-\\+][0-9]+)?|[\\-\\+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*|[\\-\\+]?\\.(?:inf|Inf|INF)|\\.(?:nan|NaN|NAN)))|((?:\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}|\\p{Nd}{4}-\\p{Nd}{1,2}-\\p{Nd}{1,2}(?:[Tt]|[ \\t]+)\\p{Nd}{1,2}:\\p{Nd}{2}:\\p{Nd}{2}(?:\\.\\p{Nd}*)?(?:[ \\t]*Z|[\\-\\+]\\p{Nd}{1,2}(?::\\p{Nd}{1,2})?)?))|(=)|(<<))(?=\\p{space}*(?=$|\\n)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=$|\\n)))/dgv,
+            /(?:(null|Null|NULL|~)|(y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF)|((?:[\\-\\+]?0b[0-1_]+|[\\-\\+]?0[0-7_]+|[\\-\\+]?(?:0|[1-9][0-9_]*)|[\\-\\+]?0x[0-9a-fA-F_]+|[\\-\\+]?[1-9][0-9_]*(?::[0-5]?[0-9])+))|((?:[\\-\\+]?(?:[0-9][0-9_]*)?\\.[0-9\\.]*(?:[eE][\\-\\+][0-9]+)?|[\\-\\+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*|[\\-\\+]?\\.(?:inf|Inf|INF)|\\.(?:nan|NaN|NAN)))|((?:\\p{Nd}{4}-\\p{Nd}{2}-\\p{Nd}{2}|\\p{Nd}{4}-\\p{Nd}{1,2}-\\p{Nd}{1,2}(?:[Tt]|[ \\t]+)\\p{Nd}{1,2}:\\p{Nd}{2}:\\p{Nd}{2}(?:\\.\\p{Nd}*)?(?:[ \\t]*Z|[\\-\\+]\\p{Nd}{1,2}(?::\\p{Nd}{1,2})?)?))|(=)|(<<))(?=\\p{space}*(?=\\n?$)|\\p{space}+#|\\p{space}*:(\\p{space}|(?=\\n?$)))/dgv,
         },
       ],
     },
@@ -395,7 +393,7 @@ exports[`precompile 1`] = `
         },
         {
           match:
-            /(?:!<(?:%[0-9A-Fa-f]{2}|[0-9A-Za-z\\-\\#\\;\\/\\?\\:\\@\\&\\=\\+\\$\\,_\\.\\!\\~\\*'\\(\\)\\[\\]])+>|!(?:[0-9A-Za-z\\-]*!)?(?:%[0-9A-Fa-f]{2}|[0-9A-Za-z\\-\\#\\;\\/\\?\\:\\@\\&\\=\\+\\$_\\.\\~\\*'\\(\\)])+|!)(?= |\\t|(?=$|\\n))/dgvy,
+            /(?:!<(?:%[0-9A-Fa-f]{2}|[0-9A-Za-z\\-\\#\\;\\/\\?\\:\\@\\&\\=\\+\\$\\,_\\.\\!\\~\\*'\\(\\)\\[\\]])+>|!(?:[0-9A-Za-z\\-]*!)?(?:%[0-9A-Fa-f]{2}|[0-9A-Za-z\\-\\#\\;\\/\\?\\:\\@\\&\\=\\+\\$_\\.\\~\\*'\\(\\)])+|!)(?= |\\t|(?=\\n?$))/dgvy,
           name: "storage.type.tag-handle.yaml",
         },
         { match: /\\P{space}+/dgv, name: "invalid.illegal.tag-handle.yaml" },