ovisocr2-mxfp4-mlx

MLX MX FP4 (micro-scaled 4-bit float) quantization of ATH-MaaS/OvisOCR2, converted with mlx-vlm.

OvisOCR2 is a compact 0.8B end-to-end document-parsing model (Qwen3.5-0.8B backbone). Given a document page image, it generates a Markdown transcription in natural reading order — text, formulas (LaTeX), tables (HTML), and visual regions (bounding-box image tags). It scores 96.58 on OmniDocBench v1.6, state of the art for an end-to-end model, and an Avg3 of 75.06 on PureDocBench.

Architecture

Base model	OvisOCR2 (Qwen3.5-VL arch, Qwen3_5ForConditionalGeneration, ~0.8B params)
Quantization	MX FP4 (micro-scaled 4-bit float)
Bits per weight	5.643
Disk size	599 MB (from ~1.6 GB bf16)
Format	MLX (safetensors)

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "sahilchachra/ovisocr2-mxfp4-mlx"
model, processor = load(model_path, trust_remote_code=True)
config = load_config(model_path, trust_remote_code=True)

prompt = (
    "\nExtract all readable content from the image in natural human reading order "
    "and output the result as a single Markdown document. For charts or images, "
    'represent them using an HTML image tag: <img src="/sahilchachra/ovisocr2-mxfp4-mlx/resolve/main/images/bbox_%7Bleft%7D_%7Btop%7D_%7Bright%7D_%7Bbottom%7D.jpg" />, '
    "where left, top, right, bottom are bounding box coordinates scaled to [0, 1000). "
    "Format formulas as LaTeX. Format tables as HTML: <table>...</table>. Transcribe all "
    "other text as standard Markdown. Preserve the original text without translation or paraphrasing."
)
formatted = apply_chat_template(processor, config, prompt, num_images=1, enable_thinking=False)
out = generate(model, processor, formatted, image="page.png", max_tokens=2048, verbose=False)
print(out.text if hasattr(out, "text") else out)

CLI:

mlx_vlm.generate --model sahilchachra/ovisocr2-mxfp4-mlx \
  --prompt "Extract all readable content from the image as Markdown." \
  --image page.png --max-tokens 2048

Document-parsing accuracy (15-page held-out eval)

15 pages sampled (seed=42, English, single-column, >500 chars of text) from opendatalab/OmniDocBench — the public benchmark OvisOCR2 reports its headline scores on (96.58 on OmniDocBench v1.6). Ground truth is the concatenation of each page's annotated text/title/list/table/equation regions in reading order. Each variant ran the model's own documented OCR prompt (mlx_vlm.generate, greedy, max_tokens=2048), and outputs were compared against ground truth with two metrics: character-level similarity (difflib.SequenceMatcher) and word recall (fraction of ground-truth words present in the output).

Variant	Bits/weight	Disk size	Mean similarity	Mean word recall	Degenerate outputs	Agreement with FP16	Eval time (15 pages)
fp16	16	1.6 GB	63.4%	92.9%	0/15	—	212.4s
mxfp4	5.643	599 MB	63.2%	93.1%	0/15	96.5%	138.1s
int4	5.863	622 MB	63.5%	93.2%	0/15	96.1%	140.7s
mxfp8	9.168	958 MB	63.0%	92.9%	0/15	97.7%	166.8s
int8	9.389	980 MB	63.9%	93.1%	0/15	99.2%	165.1s

• Word recall (~93%) is the more reliable signal here — character-level similarity is pulled down by cosmetic LaTeX-formatting differences (e.g. the model emits $...$ compact math, ground truth uses spaced $ ... $ with \left\{/\right\}) even when the transcribed content is correct. Spot-checking the lowest-similarity page (an academic-paper equation block, sim≈3%) confirmed the model's output was in fact a faithful, correctly ordered transcription — the metric penalizes LaTeX style, not content.
• Zero degenerate outputs (no repetition loops, no empty/near-empty generations) across all 75 generations (15 pages × 5 variants, including the fp16 baseline).
• All four quantized variants agree with FP16's output at the character level 96–99% of the time — quantization barely perturbs the transcription, even at 4-bit (both mxfp4 and int4).
• 4-bit vs 8-bit is a wash on quality here: mxfp4/int4 score essentially the same word recall as mxfp8/int8 on this OCR task, while being ~35% faster and roughly 40% smaller on disk — for OvisOCR2 specifically, the 4-bit variants are the better default.
• mxfp4 and int4 are nearly identical to each other in every metric; pick whichever fits your serving stack (MX FP4 vs plain affine int4) — there's no accuracy reason to prefer one.

Other MLX variants

• ovisocr2-mxfp4-mlx
• ovisocr2-int4-mlx
• ovisocr2-mxfp8-mlx
• ovisocr2-int8-mlx

Credits

• Base model: ATH-MaaS/OvisOCR2
• Eval data: opendatalab/OmniDocBench
• Quantization tooling: mlx-vlm