ArmorOCR-GGUF

This is the GGUF quantized release of ArmorOCR — a two-stage framework for grounded adversarial OCR perception built on Qwen3-VL-8B-Instruct. It provides two quantization tiers, Q8_0 and Q4_K_M, for both the main model and its vision projector (mmproj), intended to be served with llama.cpp.

📖 For training details, the AdvSpot benchmark, and the full evaluation pipeline, please visit the GitHub repo.

Quickstart

Build llama.cpp with CUDA support, then serve a tier with llama-server + --mmproj:

# 1) Build llama.cpp (CUDA)
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release
export LLAMA_BIN=$(pwd)/build/bin
pip install requests tqdm

# 2) Start the OpenAI-compatible server (Q8_0 here; use Q4_K_M likewise)
bash serve_gguf.sh Q8_0 8080

# 3) Run inference against the local server (mirrors the ArmorOCR quickstart)
import base64, requests

with open("path/to/image.png", "rb") as f:
    img_url = f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode()}"

resp = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
    "model": "ArmorOCR-GGUF",
    "messages": [{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": img_url}},
        {"type": "text", "text": ("Please identify the text in the image. "
                                  "Put your reasoning inside <analyze></analyze> "
                                  "and your final recognized text inside <answer></answer>.")},
    ]}],
    "temperature": 0.0,
    "max_tokens": 1024,
}, timeout=600)
print(resp.json()["choices"][0]["message"]["content"])

serve_gguf.sh is included in this repo for convenience.

Evaluation on AdvSpot

Region-grounded VQA accuracy and IoU on the AdvSpot benchmark. The two GGUF tiers were evaluated on a single A100 GPU with llama-server; the base column reports the original ArmorOCR results from the paper (evaluated on PPU, same data).

Category	Sub-type	base	Q8_0	Q4_K_M
Spatial Manipulation	Rotated Text	56.7	60.0	56.7
	Mirrored Text	60.0	56.7	50.0
	Tiny Text	56.7	63.3	63.3
Glyph Variation	Stylized Text	30.0	20.0	20.0
	Handwritten Text	63.3	60.0	53.3
Imaging Degradation	Capture Artifacts	60.0	60.0	56.7
	Post-processing	56.7	63.3	66.7
Contextual Blending	Low Contrast	51.4	54.3	48.6
	AIGC Fusion	75.0	77.5	72.5
	Pattern Overlay	48.6	45.7	42.9
Visual Encoding	Symbol Encoding	52.5	52.5	52.5
	Dot Encoding	53.3	56.7	50.0
	Line Encoding	60.0	66.7	66.7
Avg. Acc.		55.7	56.9	54.2
Avg. IoU		63.3	58.6	57.2

The quantized checkpoints retain accuracy close to the original ArmorOCR (Q8_0 is marginally higher on Acc), with a slight drop in IoU — consistent with quantizing the vision encoder/projector.

License

Released under the Apache License 2.0. Use is additionally subject to the license and acceptable-use policy of the base model Qwen/Qwen3-VL-8B-Instruct.

Citation

@misc{cao2026armorocrgroundedadversarialvisual,
      title={ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation}, 
      author={Linhan Cao and Siyuan Li and Jun Lan and Liangbo He and Guannan Li and Xiaolei Huang and Jun Jia and Shuheng Zhou and Huijia Zhu and Weiqiang Wang and Wei Sun},
      year={2026},
      eprint={2608.20122},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.20122}, 
}