PaddleOCR-VL-1.6 — 8-bit (MLX)

An 8-bit, group-size-64 MLX quantization of PaddlePaddle/PaddleOCR-VL-1.6, for fast on-device inference on Apple Silicon via mlx-vlm. The architecture is unchanged — this repo only re-quantizes the original weights.

Model


Base model	`PaddlePaddle/PaddleOCR-VL-1.6` (~0.9B: ERNIE-4.5-0.3B decoder + NaViT vision encoder)
Quantization	8-bit, group size 64, affine (≈9.6 bits/weight effective)
Format	MLX `safetensors`
Size	~1.1 GB
Task	image → text (OCR / document parsing)

Why 8-bit

8-bit keeps the decoder stable on harder, lower-resource scripts, where 4-bit quantization is more prone to slipping into repetition / hallucination loops. On easy text (Latin / Cyrillic / CJK) 4-bit and 8-bit are equivalent, so 8-bit is the better trade for broad multilingual OCR at a modest size increase.

Conversion

Produced with mlx-vlm 0.6.3:

python -m mlx_vlm.convert \
  --hf-path PaddlePaddle/PaddleOCR-VL-1.6 \
  --mlx-path PaddleOCR-VL-1.6-8bit \
  -q --q-bits 8 --q-group-size 64 --q-mode affine

Usage

Load with mlx-vlm; see the base model card for the OCR prompt / chat template and supported languages.

from mlx_vlm import load
model, processor = load("huggingfinger0/PaddleOCR-VL-1.6-8bit")

License & attribution

Apache-2.0, inherited from the base model. All credit for the model goes to the PaddlePaddle / PaddleOCR team — this repository only provides an MLX-quantized copy of their released weights.