typhoon-ai/typhoon-ocr2-2b-MLX-4bit

Typhoon OCR V2 2B MLX 4bit

Typhoon OCR V2 2B converted to MLX with mlx-vlm, 4-bit affine, group size 64.

Typhoon OCR V2 2B is a document parser for Thai and English. Given an image of a page, it returns the page's layout elements in reading order, each with a category, a bounding box and its text. Tables come out as HTML, formulas as LaTeX and other text as Markdown. It also answers questions about a document and extracts key information as JSON. The main model card has the training details, full benchmark results, examples and limitations.

Use of this model is subject to the OpenTyphoon Terms and Conditions.

Accuracy

Weights	Size	MDPBench ↑	MDPBench Thai ↑
Original weights (bf16, vLLM)	–	79.8	81.3
typhoon-ai/typhoon-ocr2-2b-MLX-bf16 (bfloat16, no quantization)	4.45 GB	78.9	80
typhoon-ai/typhoon-ocr2-2b-MLX-8bit (8-bit affine, group size 64)	2.68 GB	79.3	80.6
this repo (4-bit affine, group size 64)	1.74 GB	73.3	70.5

MDPBench has 320 Thai and English pages, both digital and photographed. We ran each MLX version through mlx_vlm.server (mlx 0.32.4 (mlx-cuda-13), mlx-vlm 0.7.12) with the prompt below, greedy decoding, thinking off and images of up to 16.7 MP, the same settings as the vLLM numbers in the first row. We do not have Apple hardware for this, so these scores come from the Linux CUDA build of MLX on an NVIDIA H100. The weights are the same on a Mac, but we have not checked its outputs. We recommend the 8-bit version: it scored as well as bf16 (79.3 against 78.9) at about half the size. Avoid the 4-bit version for Thai documents: its Thai score drops by about 10 points (70.5 against 80.6 for 8-bit). Differences of a few tenths of a point between versions are within run-to-run variation.

Usage with mlx-vlm

pip install -U mlx-vlm

The repo's processor config limits images to 4 MP. Pass --max-pixels 16777216 (16.7 MP), as in our evaluation; small text is read more accurately at that size.

Server

mlx_vlm.server --model typhoon-ai/typhoon-ocr2-2b-MLX-4bit --port 8080 \
  --min-pixels 2048 --max-pixels 16777216 --max-tokens 16384

mlx-vlm leaves thinking off unless you start the server with --enable-thinking; keep it off for this model. Call the OpenAI-compatible endpoint with PROMPT and the helpers from the next section:

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="EMPTY")

with open("page.jpg", "rb") as f:
    image_url = "data:image/jpeg;base64," + base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="typhoon-ai/typhoon-ocr2-2b-MLX-4bit",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": image_url}},
        {"type": "text", "text": PROMPT},
    ]}],
    temperature=0.0,
    max_tokens=16384,
)
elements = parse_elements(resp.choices[0].message.content)
print(to_markdown(elements))

Python

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("typhoon-ai/typhoon-ocr2-2b-MLX-4bit")
prompt = apply_chat_template(processor, model.config, PROMPT, num_images=1)
result = generate(model, processor, prompt, image=["page.jpg"], max_tokens=16384, temperature=0.0,
                  min_pixels=2048, max_pixels=16777216)
print(to_markdown(parse_elements(result.text)))

Prompt and output

The model was trained on a fixed set of prompts. For full-page parsing, use this prompt exactly as written:

PROMPT = """- Extract layout information from the provided PDF image.
- For each layout element, output its bbox, category, and the text content within the bbox.
- Bbox format: [x1, y1, x2, y2].
- Allowed layout categories: ['header', 'title', 'text', 'figure', 'table', 'formula', 'figure_caption', 'table_caption', 'formula_caption', 'figure_footnote', 'table_footnote', 'page_footnote', 'footer'].
- Text extraction and formatting:
  1) For 'figure', the text field must be an empty string.
  2) For 'formula', format text as LaTeX.
  3) For 'table', format text as HTML.
  4) For all other categories (e.g., text, title), format text as Markdown.
- The output text must be exactly the original text from the image, with no translation or rewriting.
- Sort all layout elements in human reading order.
- Final output must be a single JSON object."""

The model returns a JSON list of elements inside a json code block. Bounding boxes use a 0–1000 grid relative to the page width and height. These helpers parse the output and turn it into Markdown:

import json, re

def parse_elements(raw: str) -> list[dict]:
    """Parse the model output: a JSON list of elements inside a ```json fence."""
    m = re.search(r"```json\n(.*?)(?:\n```|$)", raw, re.DOTALL)
    text = (m.group(1) if m else raw).strip()
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        # Output stopped at the token limit: drop the last, incomplete element.
        cut = text.rfind('{"bbox":')
        return json.loads(text[:cut].rstrip().rstrip(",") + "]")

def to_markdown(elements: list[dict], keep_header_footer: bool = False) -> str:
    skip = set() if keep_header_footer else {"header", "footer", "page_footnote"}
    return "\n\n".join(e["text"] for e in elements if e.get("text") and e["category"] not in skip)

For questions about a document, send the question as the text prompt instead. For key-information extraction and the other tasks, see the main model card.

Limitations

• Quantization costs some accuracy; compare the scores above before picking a smaller version.
• Scores were measured with MLX on Linux (CUDA), not on Apple silicon.
• The model can fall into repetition loops on dense multi-column pages. parse_elements above keeps the complete elements when that happens.
• All limitations of the original model apply.

Support

• Discord: https://discord.gg/us5gAYmrxw
• X (Twitter): https://twitter.com/opentyphoon