axteam/PaddleOCR-VL-GGUF

PaddleOCR-VL · GGUF (llama.cpp)

GGUF quantizations of PaddlePaddle/PaddleOCR-VL for llama.cpp / Ollama / LM Studio. Produced by axteam.

File	Size
PaddleOCR-VL-Q4_K_M.gguf	0.3 GB
PaddleOCR-VL-Q8_0.gguf	0.5 GB

• Q8_0 — 8-bit, highest GGUF quality, closest to the original.
• Q4_K_M — 4-bit (k-quant medium), ~half the size, small quality trade-off.

Converted with llama.cpp convert_hf_to_gguf.py (recognized as paddleocr (ERNIE-4.5 text backbone)); the original chat_template is embedded. The vision encoder is not included in these GGUF files — this is the text-backbone LLM only, so it serves text generation, not the full OCR/image pipeline. For end-to-end OCR use the original model or the compressed-tensors builds with vLLM.

Usage

llama-cli -hf axteam/PaddleOCR-VL-GGUF

---

한국어 요약

PaddlePaddle/PaddleOCR-VL 의 GGUF 양자화(llama.cpp/Ollama/LM Studio). 작성: axteam.

• Q8_0(8비트, 최고품질) · Q4_K_M(4비트, 절반 크기).
• llama.cpp convert_hf_to_gguf.py 로 변환(paddleocr (ERNIE-4.5 text backbone) 인식), chat_template 내장.
• ⚠️ 비전 인코더 미포함 — 텍스트 백본 LLM 전용입니다. 전체 OCR 파이프라인은 원본 또는 vLLM용 compressed-tensors 빌드를 쓰세요.