michalland/OvisOCR3-GGUF

OvisOCR3 — GGUF (llama.cpp)

Unofficial llama.cpp conversion of ATH-MaaS/OvisOCR3, a unified 0.8B multimodal model (Qwen3.5-0.8B base) for OCR and document understanding: document parsing, general OCR, key information extraction, document figure captioning, and cross-page table reconstruction in a single model, selected via task prompts.

Converted with convert_hf_to_gguf.py (small local patch: the upstream converter asserted on this checkpoint's explicit mtp_num_hidden_layers: 0; patched to honor an explicit zero — the model has no MTP/nextn head) and quantized with llama-quantize. Tested on llama.cpp b11719 (Vulkan, Windows).

Files

File	quant	size
OvisOCR3-Q8_0.gguf	Q8_0	774 MB
mmproj-OvisOCR3-F16.gguf	F16 vision projector	196 MB

The mmproj is required for image input and stays F16 (vision towers are sensitive to quantization).

Quick start

llama-server \
  --hf-repo michalland/OvisOCR3-GGUF \
  --hf-file OvisOCR3-Q8_0.gguf \
  --mmproj mmproj-OvisOCR3-F16.gguf \
  --jinja --ctx-size 16384 --temp 0

Notes:

• --chat-template-kwargs "{\"enable_thinking\": false}" is recommended so OCR output never spends tokens on reasoning (the chat template ships an enable_thinking branch).
• For grounding/bounding-box tasks add --image-min-tokens 1024 (see llama.cpp#16842).
• Native context: 262,144 (16K is ample for page OCR).

Usage — task prompts

Capabilities are selected with the official prompts from the upstream repo's prompts/ directory (e.g. plain_ocr.txt, document_markdown.txt, document_layout.txt, kie_fields.txt, figure_captioning.txt, crosspage_table.txt, formula_recognition.txt, region_ocr.txt, reference_grounding.txt, ...). Send the image plus the task prompt as the user message; the model outputs the contract the prompt specifies (plain text, Markdown with LaTeX/HTML tables, JSON, bboxes, ...).

Example (plain OCR, OpenAI-compatible endpoint):

{
  "messages": [{
    "role": "user",
    "content": [
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
      {"type": "text", "text": "Read all the text in the image. Directly output the content.\n\nRules:\n1. Preserve the original text exactly without translation, paraphrasing, correction, or completion.\n2. Follow the natural human reading order.\n3. Output plain text only. Use spaces and line breaks only to preserve text separation and reading order.\n4. Do not output Markdown, HTML, JSON, bounding boxes, category labels, explanations, or code fences."}
    ]
  }],
  "temperature": 0
}

Benchmark reference (from the model card)

Model	OmniDocBench1.6	MDPBench	PureDocBench
OvisOCR2	96.58	69.00	75.10
PaddleOCR-VL-1.6	96.34	78.90	65.44
Xiaomi-OCR-0	96.83	73.00	74.60
OvisOCR3	96.74	88.80	82.31

License

Apache-2.0 (inherited from the base model). Conversion released under the same license.