Repository: shikijs/shiki (ts)

Issue:
Feature proposal: concat spans with the same style
### Clear and concise description of the problem

Here is a snippet of Shiki output (yaml, `min-light`):

```html
<span class="line">
  <span style="color:#D32F2F;">  image</span>
  <span style="color:#D32F2F;">:</span>
  <span style="color:#22863A;"> node:20-alpine</span>
</span>
```

As you can see, `  image` and `:` spans have the same color. It doesn't make sense to render them as a separate `<span>`s, and a single one `  image:` would be sufficient.



### Suggested solution

Concatenate consecutive spans with the same styling:

```html
<span class="line">
  <span style="color:#D32F2F;">  image:</span>
  <span style="color:#22863A;"> node:20-alpine</span>
</span>
```

### Alternative

_No response_

### Additional context

This change could significantly reduce amount of generated HTML and DOM nodes, which is clearly a performance benefit in web.

The gain varies depending on a theme. The less variety of colors there is, the more compaction effect there will be.

### Validations

- [x] Follow our [Code of Conduct](https://github.com/shikijs/shiki/blob/main/CODE_OF_CONDUCT.md)
- [x] Read the [Contributing Guide](https://github.com/shikijs/shiki/blob/main/CONTRIBUTING.md).
- [x] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Pull request:
feat(core): add option to merge consecutive tokens with same style (#972)
<!-- DO NOT IGNORE THE TEMPLATE!

Thank you for contributing!

Before submitting the PR, please make sure you do the following:

- Read the [Contributing Guide](https://github.com/shikijs/shiki/blob/main/CONTRIBUTING.md).
- Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
- Provide a description in this PR that addresses **what** the PR is solving, or reference the issue that it solves (e.g. `fixes #123`).
- Ideally, include relevant tests that fail without this PR but pass with it.

-->

### Description

<!-- Please insert your description here and provide especially info about the "what" this PR is solving -->
This PR addresses issue #972 by introducing a new option `mergeSameStyleTokens` in `CodeToHastOptions`. This option allows users to merge consecutive tokens with identical styles, reducing the number of DOM nodes and improving rendering performance.

### Linked Issues

fixes #972 

### Additional context

- Added `mergeSameStyleTokens` option to `CodeToHastOptions` with a default value of `false`.
- Implemented `mergeAdjacentStyledTokens` function to merge tokens with the same style.
- Updated `codeToHast` to apply token merging based on the new option.
- Documented the new option in the code comments.
- The new option is disabled by default, ensuring no breaking changes.
- When enabled, it optimizes the generated HTML by reducing `<span>` elements, which can significantly improve performance for large code blocks.
<!-- e.g. is there anything you'd like reviewers to focus on? -->


Patch:
diff --git a/docs/.vitepress/components/HomeDemo.vue b/docs/.vitepress/components/HomeDemo.vue
index c78fdc19..94f47acd 100644
--- a/docs/.vitepress/components/HomeDemo.vue
+++ b/docs/.vitepress/components/HomeDemo.vue
@@ -1,5 +1,5 @@
 <template>
-  <div class="container max-w-250 mx-a my-10 min-h-250 vp-doc p4" of-hidden>
+  <div class="container max-w-250 mx-a my-10 vp-doc p4" of-hidden>
     <ShikiMiniPlayground />
   </div>
 </template>
diff --git a/packages/colorized-brackets/package.json b/packages/colorized-brackets/package.json
index 7fd9a361..f43ef49f 100644
--- a/packages/colorized-brackets/package.json
+++ b/packages/colorized-brackets/package.json
@@ -2,7 +2,7 @@
   "name": "@shikijs/colorized-brackets",
   "type": "module",
   "version": "3.2.2",
-  "description": "Collective of common transformers transformers for Shiki",
+  "description": "VSCode-style colorized brackets transformer for Shiki",
   "author": "Michael Moore <mscottmoore@pm.me>",
   "license": "MIT",
   "homepage": "https://github.com/shikijs/shiki#readme",
diff --git a/packages/colorized-brackets/src/colorizeBracketTokens.ts b/packages/colorized-brackets/src/colorizeBracketTokens.ts
index 25b69f55..6d9f88fd 100644
--- a/packages/colorized-brackets/src/colorizeBracketTokens.ts
+++ b/packages/colorized-brackets/src/colorizeBracketTokens.ts
@@ -107,7 +107,8 @@ function assignColorToToken(
       = shikiOptions
     const styles: Record<string, string> = typeof token.htmlStyle === 'string'
       ? {}
-      : token.htmlStyle || {}
+      // clone to make sure we're not mutating a shared style object
+      : structuredClone(token.htmlStyle ?? {})
 
     for (const [colorName, theme] of Object.entries(shikiOptions.themes)) {
       const themeName = typeof theme === 'string' ? theme : theme?.name
@@ -143,7 +144,7 @@ function getColor(
   const colors
     = themeName == null
       ? DEFAULT_BRACKETS_COLORS
-      : themes[themeName] ?? builtInThemes[themeName] ?? DEFAULT_BRACKETS_COLORS
+      : getThemeColors(themeName, themes) ?? getThemeColors(themeName, builtInThemes) ?? DEFAULT_BRACKETS_COLORS
 
   const isUnexpected = level === -1
   if (isUnexpected) {
@@ -153,3 +154,16 @@ function getColor(
     return colors[level % (colors.length - 1)]
   }
 }
+
+function getThemeColors(themeName: string, themes: Record<string, string[]>): string[] | null {
+  if (themes[themeName])
+    return themes[themeName]
+
+  // check if the start of the name matches any themes
+  // this improves compatibility with "Expressive Code", which appends unique IDs to the end of themeNames
+  const startsWithName = Object.keys(themes).sort().reverse().find(key => themeName.startsWith(key))
+  if (startsWithName)
+    return themes[startsWithName]
+
+  return null
+}
diff --git a/packages/colorized-brackets/src/index.ts b/packages/colorized-brackets/src/index.ts
index 467c7300..3cf584ef 100644
--- a/packages/colorized-brackets/src/index.ts
+++ b/packages/colorized-brackets/src/index.ts
@@ -28,7 +28,7 @@ const jinjaLikeBracketPairs: BracketPair[] = [
  *
  * @param options
  * @param options.themes - custom themes; all Shiki built-in themes are supported without additional configuration
- * @param options.bracketPairs - bracket definitions; be default [], {}, (), and <> (TS-only)
+ * @param options.bracketPairs - bracket definitions; by default [], {}, (), and <> (TS-only)
  * @param options.langs - language-specific overrides for themes and bracketPairs
  * @returns Shiki transformer
  */
diff --git a/packages/core/src/highlight/code-to-hast.ts b/packages/core/src/highlight/code-to-hast.ts
index 68b3767d..5ace2996 100644
--- a/packages/core/src/highlight/code-to-hast.ts
+++ b/packages/core/src/highlight/code-to-hast.ts
@@ -47,6 +47,7 @@ export function codeToHast(
 
   const {
     mergeWhitespaces = true,
+    mergeSameStyleTokens = false,
   } = options
 
   if (mergeWhitespaces === true)
@@ -54,6 +55,10 @@ export function codeToHast(
   else if (mergeWhitespaces === 'never')
     tokens = splitWhitespaceTokens(tokens)
 
+  if (mergeSameStyleTokens) {
+    tokens = mergeAdjacentStyledTokens(tokens)
+  }
+
   const contextSource = {
     ...transformerContext,
     get source() {
@@ -306,3 +311,35 @@ function splitWhitespaceTokens(tokens: ThemedToken[][]): ThemedToken[][] {
     })
   })
 }
+
+function mergeAdjacentStyledTokens(tokens: ThemedToken[][]): ThemedToken[][] {
+  return tokens.map((line) => {
+    const newLine: ThemedToken[] = []
+    for (const token of line) {
+      if (newLine.length === 0) {
+        newLine.push({ ...token })
+        continue
+      }
+
+      const prevToken = newLine[newLine.length - 1]
+      const prevStyle = prevToken.htmlStyle || stringifyTokenStyle(getTokenStyleObject(prevToken))
+      const currentStyle = token.htmlStyle || stringifyTokenStyle(getTokenStyleObject(token))
+      const isPrevDecorated = prevToken.fontStyle && (
+        (prevToken.fontStyle & FontStyle.Underline)
+        || (prevToken.fontStyle & FontStyle.Strikethrough)
+      )
+      const isDecorated = token.fontStyle && (
+        (token.fontStyle & FontStyle.Underline)
+        || (token.fontStyle & FontStyle.Strikethrough)
+      )
+
+      if (!isPrevDecorated && !isDecorated && prevStyle === currentStyle) {
+        prevToken.content += token.content
+      }
+      else {
+        newLine.push({ ...token })
+      }
+    }
+    return newLine
+  })
+}
diff --git a/packages/types/src/options.ts b/packages/types/src/options.ts
index ef989b85..588329b8 100644
--- a/packages/types/src/options.ts
+++ b/packages/types/src/options.ts
@@ -142,6 +142,14 @@ export interface CodeToHastOptionsCommon<Languages extends string = string>
    */
   mergeWhitespaces?: boolean | 'never'
 
+  /**
+   * Merge consecutive tokens with the same style to reduce the number of DOM nodes.
+   * This can improve rendering performance but may affect the structure of the output.
+   *
+   * @default false
+   */
+  mergeSameStyleTokens?: boolean
+
   /**
    * The structure of the generated HAST and HTML.
    *


Test patch:
diff --git a/packages/colorized-brackets/test/dual-themes.test.ts b/packages/colorized-brackets/test/dual-themes.test.ts
index 4d635b5a..aba8f717 100644
--- a/packages/colorized-brackets/test/dual-themes.test.ts
+++ b/packages/colorized-brackets/test/dual-themes.test.ts
@@ -16,7 +16,7 @@ describe('dual themes', async () => {
   })
 
   it('light and dark', () => {
-    const htmlStr = highlighter.codeToHtml('{}', {
+    const htmlStr = highlighter.codeToHtml('[({})](', {
       lang,
       themes: { light: 'light-plus', dark: 'dark-plus' },
       transformers: [
@@ -28,7 +28,14 @@ describe('dual themes', async () => {
         }),
       ],
     })
-    expect(htmlStr).toContain('<span style="color:Y;--shiki-dark:y">{</span>')
+
+    expect(htmlStr).toContain('<span style="color:Y;--shiki-dark:y">[</span>')
+    expect(htmlStr).toContain('<span style="color:P;--shiki-dark:p">(</span>')
+    expect(htmlStr).toContain('<span style="color:B;--shiki-dark:b">{</span>')
+    expect(htmlStr).toContain('<span style="color:B;--shiki-dark:b">}</span>')
+    expect(htmlStr).toContain('<span style="color:P;--shiki-dark:p">)</span>')
+    expect(htmlStr).toContain('<span style="color:Y;--shiki-dark:y">]</span>')
+    expect(htmlStr).toContain('<span style="color:R;--shiki-dark:r">(</span>')
   })
 
   it('custom prefix', () => {
diff --git a/packages/core/test/hast.test.ts b/packages/core/test/hast.test.ts
index ba0bf4d9..2a802526 100644
--- a/packages/core/test/hast.test.ts
+++ b/packages/core/test/hast.test.ts
@@ -94,3 +94,44 @@ it('render whitespace', async () => {
       <span class="line"><span style="color:#59873A">		tab</span><span style="color:#999999">()</span></span></code></pre>"
     `)
 })
+
+describe('merge same style', () => {
+  it('merges adjacent tokens with same style', async () => {
+    using shiki = await createHighlighter({
+      themes: ['min-light'],
+      langs: ['yaml'],
+    })
+
+    const code = 'name: CI'
+    const html = await shiki.codeToHtml(code, {
+      lang: 'yaml',
+      theme: 'min-light',
+      mergeSameStyleTokens: true,
+    })
+
+    expect(html).toMatchInlineSnapshot(`"<pre class="shiki min-light" style="background-color:#ffffff;color:#24292eff" tabindex="0"><code><span class="line"><span style="color:#D32F2F">name:</span><span style="color:#22863A"> CI</span></span></code></pre>"`)
+  })
+
+  it('does not merge tokens with decorations', async () => {
+    using shiki = await createHighlighter({
+      themes: ['min-light'],
+      langs: ['yaml'],
+    })
+
+    const code = 'name: CI'
+    const html = await shiki.codeToHtml(code, {
+      lang: 'yaml',
+      theme: 'min-light',
+      mergeSameStyleTokens: true,
+      decorations: [
+        {
+          start: { line: 0, character: 0 },
+          end: { line: 0, character: 4 },
+          properties: { class: 'highlighted-word' },
+        },
+      ],
+    })
+
+    expect(html).toMatchInlineSnapshot(`"<pre class="shiki min-light" style="background-color:#ffffff;color:#24292eff" tabindex="0"><code><span class="line"><span style="color:#D32F2F" class="highlighted-word">name</span><span style="color:#D32F2F">:</span><span style="color:#22863A"> CI</span></span></code></pre>"`)
+  })
+})