AnandSingh/LightOnOCR-3-4B-bf16

LightOnOCR-3-4B — MLX (Apple Silicon)

BF16 weights of lightonai/LightOnOCR-3-4B converted to MLX for native inference on Apple Silicon via mlx-vlm (≥ 0.6.5).

LightOnOCR-3-4B is a Qwen3.5-based vision-language OCR model: it transcribes documents, tables, receipts, forms, charts and math to Markdown, and in grounding mode returns labeled bounding boxes for every block, short descriptions of images, and the underlying data of charts as HTML tables.

Install

pip install -U "mlx-vlm>=0.6.5"

The architecture is the stock qwen3_5 multimodal model already shipped in mlx-vlm, so this is a pure weight conversion — no patches required.

Usage

Transcription (default / "plain")

mlx_vlm.generate \
  --model AnandSingh/LightOnOCR-3-4B-bf16 \
  --image ./tests/images/receipt.jpeg \
  --prompt "" \
  --max-tokens 2048 --temperature 0.2 --top-p 0.9

Grounding (labels + bounding boxes)

mlx_vlm.generate \
  --model AnandSingh/LightOnOCR-3-4B-bf16 \
  --image page.png \
  --prompt "grounding" \
  --max-tokens 4096 --temperature 0.2 --top-p 0.9

Python

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("AnandSingh/LightOnOCR-3-4B-bf16")
prompt = apply_chat_template(processor, model.config, "grounding", num_images=1)
result = generate(model, processor, prompt, image=["page.png"],
                  max_tokens=4096, temperature=0.2, top_p=0.9)
print(result.text)

A lighter, faster build lives at AnandSingh/LightOnOCR-3-4B-4bit (3 GB, ~37 tok/s on M4) — recommended for most use. This BF16 build is the full-precision reference (9.1 GB, ~11 tok/s), validated for numerical parity against the original PyTorch checkpoint.

See run_ocr.py in the LightOnOCR-3 conversion repo for a runnable plain/grounding wrapper.

Prompt modes

Mode	Prompt	Output
Transcription	"" (empty)	All page text as Markdown
Grounding	"grounding"	Each block prefixed with  (0–1000 page coords)

Other instructions are out of distribution — use the empty prompt or grounding. Thinking is disabled by default (the model was trained/evaluated that way), which apply_chat_template sets automatically via enable_thinking=False.

Preprocessing tips

• Render PDFs at 400 DPI, then resize the longest edge to 2048 px (aspect ratio preserved). The bundled image processor (Qwen3-VL smart-resize) handles patch/merge sizing internally.

Provenance

Converted with mlx_vlm.convert from the original BF16 transformers checkpoint. Weight keys were remapped by mlx-vlm's qwen3_5 sanitizer (model.language_model.* → language_model, model.visual.* → vision_tower), Conv1d axis flip, 5-D patch-embed transpose, and the +1.0 RMSNorm offset. Validated for numerical parity against the PyTorch reference (see repo README).

• Source: https://huggingface.co/lightonai/LightOnOCR-3-4B
• License: Apache-2.0