npm package: @prantlf/semidown
Description: Semi-incremental markdown parser and renderer, optimized for LLM output.
Author: Chuanqisun Sun
Latest version: 5.1.0

# Semidown

A semi-incremental streaming markdown parser and renderer for the web, designed for handling LLM streaming out.

[Try the live demo!](https://prantlf.github.io/semidown/)

This is a fork of the original project with the following enhancements:

* ESM, CJS and UMD exports, not minified and minified, including source maps
* Default exports load external dependencies (`marked`, `marked-shiki` and `shiki/bundle/web`)
* The `core` exports have no dependencies, neither external, nor bundled
* The `bundle` exports include the dependencies (`marked`, `marked-shiki` and `shiki/bundle/web`)
* Fixes rendering of fenced blocks with empty lines by withholding their parsing until they are complete.
* Can withhold parsing of incomplete links to reduce twitching of the render output.
* Lets override and extend parser, chunker and renderer.

## Install

```bash
npm install @prantlf/semidown
```

## Usage

Below is a simplified usage example. See `src/main.ts` for a full-featured demo with UI controls.

```typescript
import { Semidown } from "@prantlf/semidown";

const outputContainer = document.getElementById("output");
const parser = new Semidown(outputContainer);

// Simulate streaming input
const markdown = "# Hello\nThis is a **streaming** demo.";
let idx = 0;
const interval = setInterval(() => {
  if (idx < markdown.length) {
    parser.write(markdown[idx++]);
  } else {
    parser.end();
    clearInterval(interval);
  }
}, 50);
parser.on('end', () => {
  console.log("markdown processing ended");
})
```

## Customisation

If you need to customise the `Marked` instance, or do not need `Shiki` for syntax highlighting, you can use the import `@prantlf/semidown/core` instead.

```typescript
import { SemidownCore, MarkdownParserCore } from "@prantlf/semidown/core";
import { Marked } from "marked";

const targetElement = document.getElementById("output");
const marked = new Marked();
const parser = new MarkdownParserCore({ marked });
const semidown = new SemidownCore({ targetElement, parser });
```

How to prevent a flash of unfinished to finished inline elements like bold and italic text or hyperlinks with [remend](https://github.com/vercel/streamdown/tree/main/packages/remend):

```ts
import { Semidown, MarkdownStreamChunker } from "@prantlf/semidown/core";
import remend from "remend";

class MendingMarkdownStreamChunker extends MarkdownStreamChunker {
  protected emitUpdate(content, isComplete) {
    if (!isComplete) {
      content = remend(content, { katex: false });
    }
    super.emitUpdate(content, isComplete);
  }
}

const targetElement = document.getElementById("output");
const chunker = new MendingMarkdownStreamChunker();
const semidown = new Semidown({ targetElement, chunker });
```

For the full customisation, you can create instances of `chunker`, `parser` and `renderer` on your own.

```typescript
import {
  SemidownCore,
  MarkdownStreamChunker,
  MarkdownParserCore,
  HTMLRenderer
} from "@prantlf/semidown/core";
import { Marked } from "marked";

const targetElement = document.getElementById("output");
const chunker = new MarkdownStreamChunker({
  blockIdPrefix: "block-",
  withholdIncompleteFencedBlocks: true,
  withholdIncompleteLinks: false
});
const marked = new Marked();
const parser = new MarkdownParserCore({ marked });
const renderer = new HTMLRenderer({
  targetElement,
  datasetProperty: "blockId",
  completeCssClass: "md-blk-complete"
});
const semidown = new SemidownCore({ chunker, parser, renderer });
```

`SemidownCore`/`Semidown` instances trigger events once `write` or `end` methods were called to schedule another Markdown chunk to be processed. Unline events triggered by `chunker` and `renderer`, these events are triggered when the asynchronous processing was finished:

* `block-create` - one block was created
* `block-update` - one block was updated
* `block-complete` - one block was completed
* `end` - `end` was called and all blocks were completed

## Installation

You can install an NPM package by your favourite package manager, for example:

    npm i @prantlf/semidown

Or refer to it at `unpkg` on a web page:

    https://unpkg.com/@prantlf/semidown@4.1.0/dist/...

And choose a file from the table below instead of the three dots.

## Exports

Exported files in the `dist` directory, replace the three dots with the path to the `dist` directory above:

| Distributed file   | Loading statement            | Format | Minified | Dependencies | Environment   |
|:-------------------|:-----------------------------|--------|----------|--------------|---------------|
| `index.cjs`        | `import '@prantlf/semidown'` |   EJS  |    no    |   external   |      node     |
| `index.js`         | `import '@prantlf/semidown'` |   EJS  |    no    |   external   | node, browser |
| `index.min.js`     | `import '.../index.min.js'`  |   EJS  |    yes   |   external   |    browser    |
| `index.umd.js`     | `<script src=".../index.umd.js">`     |  UMD  |  no  | external |    browser    |
| `index.umd.min.js` | `<script src=".../index.umd.min.js">` |  UMD  |  yes | external |    browser    |
| `index-core.cjs`    | `import '@prantlf/semidown/core'` | EJS |  no   |     none     |      node     |
| `index-core.js`     | `import '@prantlf/semidown/core'` | EJS |  no   |     none     | node, browser |
| `index-core.min.js` | `import '.../index-core.min.js'`  | EJS |  yes  |     none     |    browser    |
| `index-core.umd.js` | `<script src=".../index-core.umd.js">` | UMD | no |   none     |    browser    |
| `index-core.umd.min.js` | `<script src=".../index-core.umd.min.js">` | UMD | yes | none | browser    |
| `index-bundle.js`         | `import '.../index-bundle.js'`     | EJS | no  | bundled |    browser    |
| `index-bundle.min.js`     | `import '.../index-bundle.min.js'` | EJS | yes | bundled |    browser    |
| `index-bundle.umd.js`     | `<script src=".../index-bundle.umd.js">`     | UMD | no  | bundled | browser |
| `index-bundle.umd.min.js` | `<script src=".../index-bundle.umd.min.js">` | UMD | yes | bundled | browser |

UMD modules are named `semidown` for being imported as an AMD dependency or found in a global namespace as IIFE.

## Key Features

- **Semi-incremental Parsing:**
  - **Block-level incremental:** As new markdown text arrives (e.g., from a stream), the parser incrementally processes and renders new blocks (paragraphs, lists, code blocks, tables, etc.) without reprocessing the entire document.
  - **Inline-level re-rendering:** Within a block, inline elements (bold, italics, links, code spans, etc.) are re-rendered as the block's content grows. This design ensures robust and correct inline rendering, even as partial input arrives.
- **Streaming Support:** Feed markdown text in arbitrary chunks (e.g., as it arrives from a network or user input) and see the output update in real time.
- **Pause, Resume, and Fast-forward:** Control the streaming process for demos, testing, or user interaction.
- **Performance & Robustness:** The semi-incremental approach balances efficient updates with correct rendering, avoiding flicker or broken formatting as content streams in.

## How It Works

- The parser maintains state as new markdown text is written.
- When a complete block is detected, it is parsed and rendered immediately.
- If a block is incomplete (e.g., a code block or list is still being typed), it is held and re-parsed as more text arrives.
- Inline elements within a block are always re-parsed on each update to ensure correctness.

## Sample App

The included demo (`src/main.ts`) provides a UI to:

- Start, pause, resume, and stop streaming markdown input
- See real-time updates as markdown is parsed and rendered
- Experiment with different markdown features (lists, code, tables, etc.)

## Design Rationale

- **Why semi-incremental?**
  - Full incremental parsing at the inline level is complex and error-prone, especially with partial tokens and nested formatting.
  - By incrementally parsing at the block level and re-rendering inlines, the library achieves a robust balance: fast updates for new blocks, and correct inline formatting as content grows.

## Contributing

In lieu of a formal styleguide, take care to maintain the existing coding style. Lint and test your code.

## License

Copyright (c) 2022-2025 Chuanqisun Sun \
Copyright (C) 2025 Ferdinand Prantl

Licensed under the [MIT License].

[MIT License]: http://en.wikipedia.org/wiki/MIT_License