Repository: shikijs/shiki
Pull request #826 (2024-11-04 16:15:17)

feat(transformers): add Style to Class transformer
fix https://github.com/shikijs/shiki/issues/824

Introduce a new transformer `transformerStyleToClass` that converts Shiki's inline styles to unique classes.

Class names are generated based on the hash value of the style object with the prefix/suffix you provide. You can put this transformer in multiple highlights passes and then get the CSS at the end to reuse the exact same styles. As Shiki doesn't handle CSS, it's on your integration to decide how to extract and apply/bundle the CSS.

For example:

```ts
import { transformerStyleToClass } from '@shikijs/transformers'
import { codeToHtml } from 'shiki'

const toClass = transformerStyleToClass({ // [!code highlight:3]
  classPrefix: '__shiki_',
})

const code = `console.log('hello')`
const html = await codeToHtml(code, {
  lang: 'ts',
  themes: {
    dark: 'vitesse-dark',
    light: 'vitesse-light',
  },
  defaultColor: false,
  transformers: [toClass], // [!code highlight]
})

// The transformer instance exposes some methods to get the CSS
const css = toClass.getCSS() // [!code highlight]

// use `html` and `css` in your app
```

HTML output:

```html
<pre class="shiki shiki-themes vitesse-dark vitesse-light __shiki_9knfln" tabindex="0"><code><span class="line">
  <span class="__shiki_14cn0u">console</span>
  <span class="__shiki_ps5uht">.</span>
  <span class="__shiki_1zrdwt">log</span>
  <span class="__shiki_ps5uht">(</span>
  <span class="__shiki_236mh3">'</span>
  <span class="__shiki_1g4r39">hello</span>
  <span class="__shiki_236mh3">'</span>
  <span class="__shiki_ps5uht">)</span>
</span></code></pre>
```

CSS output:

```css
.__shiki_14cn0u {
  --shiki-dark: #bd976a;
  --shiki-light: #b07d48;
}
.__shiki_ps5uht {
  --shiki-dark: #666666;
  --shiki-light: #999999;
}
.__shiki_1zrdwt {
  --shiki-dark: #80a665;
  --shiki-light: #59873a;
}
.__shiki_236mh3 {
  --shiki-dark: #c98a7d77;
  --shiki-light: #b5695977;
}
.__shiki_1g4r39 {
  --shiki-dark: #c98a7d;
  --shiki-light: #b56959;
}
.__shiki_9knfln {
  --shiki-dark: #dbd7caee;
  --shiki-light: #393a34;
  --shiki-dark-bg: #121212;
  --shiki-light-bg: #ffffff;
}
```



netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *shiki-next* processing.


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 7f86e71687c3610c78ff136004dc5182cf1e168d |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/shiki-next/deploys/6728f3183c0b440008d6534b |
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/shikijs/shiki/pull/826?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs) Report
Attention: Patch coverage is `93.82716%` with `5 lines` in your changes missing coverage. Please review.
> Project coverage is 94.85%. Comparing base [(`9b55bf8`)](https://app.codecov.io/gh/shikijs/shiki/commit/9b55bf8f9583cbeac6d67779533807e187dfe306?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs) to head [(`7f86e71`)](https://app.codecov.io/gh/shikijs/shiki/commit/7f86e71687c3610c78ff136004dc5182cf1e168d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs).
> Report is 1 commits behind head on main.

| [Files with missing lines](https://app.codecov.io/gh/shikijs/shiki/pull/826?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs) | Patch % | Lines |
|---|---|---|
| [...es/transformers/src/transformers/style-to-class.ts](https://app.codecov.io/gh/shikijs/shiki/pull/826?src=pr&el=tree&filepath=packages%2Ftransformers%2Fsrc%2Ftransformers%2Fstyle-to-class.ts&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs#diff-cGFja2FnZXMvdHJhbnNmb3JtZXJzL3NyYy90cmFuc2Zvcm1lcnMvc3R5bGUtdG8tY2xhc3MudHM=) | 93.75% | [5 Missing :warning: ](https://app.codecov.io/gh/shikijs/shiki/pull/826?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #826      +/-   ##
==========================================
- Coverage   94.86%   94.85%   -0.02%     
==========================================
  Files          78       79       +1     
  Lines        6608     6689      +81     
  Branches     1319     1340      +21     
==========================================
+ Hits         6269     6345      +76     
- Misses        330      335       +5     
  Partials        9        9              
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/shikijs/shiki/pull/826?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=shikijs).

netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *shiki-matsu* processing.


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 7f86e71687c3610c78ff136004dc5182cf1e168d |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/shiki-matsu/deploys/6728f3183c0b440008d65347 |

Patch:
diff --git a/docs/.vitepress/components/LanguagesList.vue b/docs/.vitepress/components/LanguagesList.vue
index 62a5f3fb..7d9e6d5b 100644
--- a/docs/.vitepress/components/LanguagesList.vue
+++ b/docs/.vitepress/components/LanguagesList.vue
@@ -1,5 +1,5 @@
 <script setup lang="ts">
-import { computed, ref } from 'vue'
+import { computed, ref, watch } from 'vue'
 import { usePlayground } from '../store/playground'
 
 const play = usePlayground()
@@ -17,6 +17,14 @@ const langs = computed(() => {
     return play.bundledLangsWeb
   return play.bundledLangsFull
 })
+watch(showModel, () => {
+  if (showModel.value) {
+    (document.scrollingElement as HTMLElement).style.overflow = 'hidden'
+  }
+  else {
+    (document.scrollingElement as HTMLElement).style.overflow = 'initial'
+  }
+})
 </script>
 
 <template>
diff --git a/docs/.vitepress/components/ShikiMiniPlayground.vue b/docs/.vitepress/components/ShikiMiniPlayground.vue
index 173c3193..09108acf 100644
--- a/docs/.vitepress/components/ShikiMiniPlayground.vue
+++ b/docs/.vitepress/components/ShikiMiniPlayground.vue
@@ -27,10 +27,10 @@ function onInput() {
 
 <template>
   <div
-    class="language-ts vp-adaptive-theme mini-playground transition-none!" shadow
+    class="language-ts vp-adaptive-theme transition-none! mini-playground" shadow
     :style="[play.preStyle, { colorScheme: currentThemeType }]"
   >
-    <div absolute z-10 p2 px3 pl5 flex="~ gap-1 items-center" left-0 top-0 right-0 border="b-solid gray/5">
+    <div sticky z-12 p2 px3 pl5 flex="~ gap-1 items-center" left-0 top-0 right-0 border="b-solid gray/5" bg-inherit>
       <div i-carbon:chevron-down op50 />
       <select v-model="play.lang" font-mono :style="play.preStyle">
         <option v-for="lang in play.allLanguages" :key="lang.id" :value="lang.id">
@@ -62,7 +62,7 @@ function onInput() {
         <div i-carbon:shuffle op50 />
       </button>
     </div>
-    <div relative mt-10 min-h-100>
+    <div relative min-h-100 float-left min-w-full>
       <span ref="highlightContainerRef" v-html="play.output" />
       <textarea
         ref="textAreaRef"
@@ -80,15 +80,11 @@ function onInput() {
 </template>
 
 <style>
-.mini-playground {
-  overflow: hidden;
-}
-
 .mini-playground select {
   background: transparent;
   color: inherit;
   min-width: 8em;
-  padding: 0 !important;
+  padding: 0px !important;
   position: relative;
 }
 .mini-playground select:focus {
diff --git a/docs/guide/transformers.md b/docs/guide/transformers.md
index 8d371def..f50ff30f 100644
--- a/docs/guide/transformers.md
+++ b/docs/guide/transformers.md
@@ -62,7 +62,7 @@ Transformers can also access markdown 'meta' strings in [supported integrations]
 
 ````markdown
 <!-- [!code word:meta=here] -->
-```html meta=here
+```html foo=bar baz-qux="qu ux"
 ````
 
 You can access the raw meta using:
@@ -71,5 +71,5 @@ You can access the raw meta using:
 
 ```ts
 options.meta
-// => { meta: 'here', __raw: 'meta=here' }
+// => { __raw: 'foo=bar baz-qux="qu ux"' }
 ```
diff --git a/docs/packages/transformers.md b/docs/packages/transformers.md
index e13db3fa..f1106e7b 100644
--- a/docs/packages/transformers.md
+++ b/docs/packages/transformers.md
@@ -348,3 +348,84 @@ Remove line breaks between `<span class="line">`. Useful when you set `display:
 
 Transform `// [\!code ...]` to `// [!code ...]`.
 Avoid rendering the escaped notation syntax as it is.
+
+---
+
+### `transformerStyleToClass`
+
+Convert Shiki's inline styles to unique classes.
+
+Class names are generated based on the hash value of the style object with the prefix/suffix you provide. You can put this transformer in multiple highlights passes and then get the CSS at the end to reuse the exact same styles. As Shiki doesn't handle CSS, it's on your integration to decide how to extract and apply/bundle the CSS.
+
+For example:
+
+```ts
+import { transformerStyleToClass } from '@shikijs/transformers'
+import { codeToHtml } from 'shiki'
+
+const toClass = transformerStyleToClass({ // [!code highlight:3]
+  classPrefix: '__shiki_',
+})
+
+const code = `console.log('hello')`
+const html = await codeToHtml(code, {
+  lang: 'ts',
+  themes: {
+    dark: 'vitesse-dark',
+    light: 'vitesse-light',
+  },
+  defaultColor: false,
+  transformers: [toClass], // [!code highlight]
+})
+
+// The transformer instance exposes some methods to get the CSS
+const css = toClass.getCSS() // [!code highlight]
+
+// use `html` and `css` in your app
+```
+
+HTML output:
+
+```html
+<pre class="shiki shiki-themes vitesse-dark vitesse-light __shiki_9knfln" tabindex="0"><code><span class="line">
+  <span class="__shiki_14cn0u">console</span>
+  <span class="__shiki_ps5uht">.</span>
+  <span class="__shiki_1zrdwt">log</span>
+  <span class="__shiki_ps5uht">(</span>
+  <span class="__shiki_236mh3">'</span>
+  <span class="__shiki_1g4r39">hello</span>
+  <span class="__shiki_236mh3">'</span>
+  <span class="__shiki_ps5uht">)</span>
+</span></code></pre>
+```
+
+CSS output:
+
+```css
+.__shiki_14cn0u {
+  --shiki-dark: #bd976a;
+  --shiki-light: #b07d48;
+}
+.__shiki_ps5uht {
+  --shiki-dark: #666666;
+  --shiki-light: #999999;
+}
+.__shiki_1zrdwt {
+  --shiki-dark: #80a665;
+  --shiki-light: #59873a;
+}
+.__shiki_236mh3 {
+  --shiki-dark: #c98a7d77;
+  --shiki-light: #b5695977;
+}
+.__shiki_1g4r39 {
+  --shiki-dark: #c98a7d;
+  --shiki-light: #b56959;
+}
+.__shiki_9knfln {
+  --shiki-dark: #dbd7caee;
+  --shiki-light: #393a34;
+  --shiki-dark-bg: #121212;
+  --shiki-light-bg: #ffffff;
+}
+```
diff --git a/packages/transformers/src/index.ts b/packages/transformers/src/index.ts
index 1d305eaa..a7f5291a 100644
--- a/packages/transformers/src/index.ts
+++ b/packages/transformers/src/index.ts
@@ -9,4 +9,5 @@ export * from './transformers/notation-highlight-word'
 export * from './transformers/remove-line-breaks'
 export * from './transformers/remove-notation-escape'
 export * from './transformers/render-whitespace'
+export * from './transformers/style-to-class'
 export * from './utils'
diff --git a/packages/transformers/src/transformers/style-to-class.ts b/packages/transformers/src/transformers/style-to-class.ts
new file mode 100644
index 00000000..771e5297
--- /dev/null
+++ b/packages/transformers/src/transformers/style-to-class.ts
@@ -0,0 +1,123 @@
+import type { ShikiTransformer } from 'shiki'
+
+export interface TransformerStyleToClassOptions {
+  /**
+   * Prefix for class names.
+   * @default '__shiki_'
+   */
+  classPrefix?: string
+  /**
+   * Suffix for class names.
+   * @default ''
+   */
+  classSuffix?: string
+  /**
+   * Callback to replace class names.
+   * @default (className) => className
+   */
+  classReplacer?: (className: string) => string
+}
+
+export interface ShikiTransformerStyleToClass extends ShikiTransformer {
+  getClassRegistry: () => Map<string, Record<string, string> | string>
+  getCSS: () => string
+  clearRegistry: () => void
+}
+
+/**
+ * Remove line breaks between lines.
+ * Useful when you override `display: block` to `.line` in CSS.
+ */
+export function transformerStyleToClass(options: TransformerStyleToClassOptions = {}): ShikiTransformerStyleToClass {
+  const {
+    classPrefix = '__shiki_',
+    classSuffix = '',
+    classReplacer = (className: string) => className,
+  } = options
+
+  const classToStyle = new Map<string, Record<string, string> | string>()
+
+  function stringifyStyle(style: Record<string, string>): string {
+    return Object.entries(style)
+      .map(([key, value]) => `${key}:${value}`)
+      .join(';')
+  }
+
+  function registerStyle(style: Record<string, string> | string): string {
+    const str = typeof style === 'string'
+      ? style
+      : stringifyStyle(style)
+    let className = classPrefix + cyrb53(str) + classSuffix
+    className = classReplacer(className)
+    if (!classToStyle.has(className)) {
+      classToStyle.set(
+        className,
+        typeof style === 'string'
+          ? style
+          : { ...style },
+      )
+    }
+    return className
+  }
+
+  return {
+    name: '@shikijs/transformers:style-to-class',
+    pre(t) {
+      if (!t.properties.style)
+        return
+      const className = registerStyle(t.properties.style as string)
+      delete t.properties.style
+      this.addClassToHast(t, className)
+    },
+    tokens(lines) {
+      for (const line of lines) {
+        for (const token of line) {
+          if (!token.htmlStyle)
+            continue
+
+          const className = registerStyle(token.htmlStyle)
+          token.htmlStyle = {}
+          token.htmlAttrs ||= {}
+          if (!token.htmlAttrs.class)
+            token.htmlAttrs.class = className
+          else
+            token.htmlAttrs.class += ` ${className}`
+        }
+      }
+    },
+    getClassRegistry() {
+      return classToStyle
+    },
+    getCSS() {
+      let css = ''
+      for (const [className, style] of classToStyle.entries()) {
+        css += `.${className}{${typeof style === 'string' ? style : stringifyStyle(style)}}`
+      }
+      return css
+    },
+    clearRegistry() {
+      classToStyle.clear()
+    },
+  }
+}
+
+/**
+ * A simple hash function.
+ *
+ * @see https://stackoverflow.com/a/52171480
+ */
+function cyrb53(str: string, seed = 0): string {
+  let h1 = 0xDEADBEEF ^ seed
+  let h2 = 0x41C6CE57 ^ seed
+  for (let i = 0, ch; i < str.length; i++) {
+    ch = str.charCodeAt(i)
+    h1 = Math.imul(h1 ^ ch, 2654435761)
+    h2 = Math.imul(h2 ^ ch, 1597334677)
+  }
+  h1 = Math.imul(h1 ^ (h1 >>> 16), 2246822507)
+  h1 ^= Math.imul(h2 ^ (h2 >>> 13), 3266489909)
+  h2 = Math.imul(h2 ^ (h2 >>> 16), 2246822507)
+  h2 ^= Math.imul(h1 ^ (h1 >>> 13), 3266489909)
+
+  return (4294967296 * (2097151 & h2) + (h1 >>> 0)).toString(36).slice(0, 6)
+}


Test patch:
diff --git a/packages/transformers/test/style-to-class.test.ts b/packages/transformers/test/style-to-class.test.ts
new file mode 100644
index 00000000..9db4d329
--- /dev/null
+++ b/packages/transformers/test/style-to-class.test.ts
@@ -0,0 +1,50 @@
+import { createHighlighter } from 'shiki'
+import { expect, it } from 'vitest'
+import { transformerStyleToClass } from '../src/transformers/style-to-class'
+
+it('transformerStyleToClass', async () => {
+  const shiki = await createHighlighter({
+    themes: ['vitesse-dark', 'vitesse-light', 'nord'],
+    langs: ['typescript'],
+  })
+
+  const transformer = transformerStyleToClass()
+
+  const code = `
+    const a = Math.random() > 0.5 ? 1 : \`foo\`
+  `.trim()
+
+  const result = shiki.codeToHtml(code, {
+    lang: 'typescript',
+    themes: {
+      dark: 'vitesse-dark',
+      light: 'vitesse-light',
+      nord: 'nord',
+    },
+    defaultColor: false,
+    transformers: [transformer],
+  })
+
+  expect(result.replace(/<span/g, '\n<span'))
+    .toMatchInlineSnapshot(`
+      "<pre class="shiki shiki-themes vitesse-dark vitesse-light nord __shiki_uywmyh" tabindex="0"><code>
+      <span class="line">
+      <span class="__shiki_223nhr">const</span>
+      <span class="__shiki_u5wfov"> a</span>
+      <span class="__shiki_26darv"> =</span>
+      <span class="__shiki_u5wfov"> Math</span>
+      <span class="__shiki_17lqoe">.</span>
+      <span class="__shiki_6u0ar0">random</span>
+      <span class="__shiki_k92bfk">()</span>
+      <span class="__shiki_26darv"> ></span>
+      <span class="__shiki_1328cg"> 0.5</span>
+      <span class="__shiki_223nhr"> ?</span>
+      <span class="__shiki_1328cg"> 1</span>
+      <span class="__shiki_223nhr"> :</span>
+      <span class="__shiki_ga6n9x"> \`</span>
+      <span class="__shiki_23isjw">foo</span>
+      <span class="__shiki_ga6n9x">\`</span></span></code></pre>"
+    `)
+
+  expect(transformer.getCSS()).toMatchInlineSnapshot(`".__shiki_223nhr{--shiki-dark:#CB7676;--shiki-light:#AB5959;--shiki-nord:#81A1C1}.__shiki_u5wfov{--shiki-dark:#BD976A;--shiki-light:#B07D48;--shiki-nord:#D8DEE9}.__shiki_26darv{--shiki-dark:#666666;--shiki-light:#999999;--shiki-nord:#81A1C1}.__shiki_17lqoe{--shiki-dark:#666666;--shiki-light:#999999;--shiki-nord:#ECEFF4}.__shiki_6u0ar0{--shiki-dark:#80A665;--shiki-light:#59873A;--shiki-nord:#88C0D0}.__shiki_k92bfk{--shiki-dark:#666666;--shiki-light:#999999;--shiki-nord:#D8DEE9FF}.__shiki_1328cg{--shiki-dark:#4C9A91;--shiki-light:#2F798A;--shiki-nord:#B48EAD}.__shiki_ga6n9x{--shiki-dark:#C98A7D77;--shiki-light:#B5695977;--shiki-nord:#ECEFF4}.__shiki_23isjw{--shiki-dark:#C98A7D;--shiki-light:#B56959;--shiki-nord:#A3BE8C}.__shiki_uywmyh{--shiki-dark:#dbd7caee;--shiki-light:#393a34;--shiki-nord:#d8dee9ff;--shiki-dark-bg:#121212;--shiki-light-bg:#ffffff;--shiki-nord-bg:#2e3440ff}"`)
+})