OvisOCR2-AWQ

AWQ (W4A16 asymmetric, group size 128) quantization of ATH-MaaS/OvisOCR2 — a 0.8B end-to-end document parsing / OCR model built on Qwen3.5-0.8B.

Made with llm-compressor: AWQModifier applied to all Linear layers of the language model. The vision tower and lm_head are kept in BF16.

• Calibration: 256 samples from HuggingFaceH4/ultrachat_200k, max sequence length 2048
• Scheme: W4A16_ASYM
• Ignored modules: lm_head, visual.* (vision tower)

Usage (vLLM)

vllm serve <this-repo> --trust-remote-code=false

Prompt for full-page OCR (same as base model):

Extract all readable content from the image in natural human reading order and output the result as a single Markdown document. Format formulas as LaTeX. Format tables as HTML: <table>...</table>. Preserve the original text without translation.

Use temperature 0.0 for OCR tasks.

Notes

Quantized entirely with open tooling; see the base model card for benchmarks, capabilities and limitations of the underlying model. Weight-only 4-bit quantization may introduce a small accuracy loss versus BF16 on dense documents.