mlx-community/LensVLM-9B-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

Apple's document-reading VLM, quantized for a Mac. A mixed-precision MLX quant of apple/LensVLM-9B. 8.0 GB on disk, down from 17.5 GB bf16.

LensVLM reads text as pictures of text. Instead of feeding a long document through the context window token by token, it renders pages as compressed images, scans them, and expands only the pages it needs back to full resolution. The language tower is Qwen3.5-9B; the vision tower is the Qwen3-VL encoder.

OptiQ quantizes the language tower to mixed 4/8-bit and keeps the vision tower at bf16 in a sidecar, so one checkpoint does both text and images.

It reads images

Run through this quantized model on Apple Silicon, MLX only, greedy decoding:

Input	Output
a red circle on white	"a red circle"
a blue square on white	"a blue square"
a green triangle on white	"a green triangle"

What it is

Property	Value
Base	apple/LensVLM-9B (Qwen3-VL encoder + Qwen3.5-9B language tower)
Language tower	per-layer 4/8-bit: 116 layers at 4-bit, 134 at 8-bit (250 total)
Vision tower	bf16, kept in optiq/optiq_vision.safetensors (333 tensors)
On disk	7.10 GB language + 0.91 GB vision
Group size	64
Context	262,144 tokens

Following llama.cpp's naming for mixed quants, the "4bit" label denotes the predominant precision, not the weighted average.

How the bit-widths were chosen

Stated plainly, because it differs from most OptiQ quants: the per-layer allocation was not measured on this model. It was transferred from mlx-community/Qwen3.5-9B-OptiQ-4bit, whose allocation came from a KL-divergence sensitivity sweep over a six-domain calibration mix (prose, reasoning, code, agent, tool-call, instructions).

That transfer is sound here because the two language towers share an architecture exactly — qwen3_5_text, 32 layers, 16 attention heads, 4 KV heads, head_dim 256, hidden 4096, vocab 248,320 — so every layer in the recipe has a counterpart with the same role and shape. All 250 tensors matched with none unmatched, which is the check that matters: an unmatched tensor would silently fall back to flat 4-bit and make this a uniform quant wearing a mixed-precision name.

What sensitivity measures is how much a layer's role in the architecture suffers from precision loss. What it cannot know is whether this model's own training moved that sensitivity around — and LensVLM was trained for a narrow, unusual job, so that caveat has more weight here than it does for a general-purpose fine-tune. If you want the allocation measured against this model, run optiq convert on it and let the sweep do the work.

What was verified

• 250/250 tensors matched the recipe, 0 unmatched.
• Vision: three shapes and three colours, read correctly through the quantized model (table above).
• Text: factual recall (Paris), arithmetic with working shown (240 km in 3 h → 80 km/h), and an iterative Fibonacci that runs correctly.
• OptiQ's release contract (artifact layout, sidecar location, metadata, mixed-precision assertions).

Not run for this model: the six-metric Capability Score. The published scores for the Qwen3.5-9B quant describe that model, not this one, and are not claimed here. A general benchmark would also be a poor fit — LensVLM is built for compressed-document reading, not for MMLU.

A note on its reasoning style

The model narrates in the idiom it was trained in. Asked a plain text question with no document attached, it will still reason about page thumbnails:

The question asks for the capital of France… Scanning the available page thumbnails, Page 1 mentions "European Union" and "France"…

Paris

The answer is right. The scaffolding around it is the document-reading habit showing through, and it is present in the base model too. Use it for what it is built for.

Use it

pip install mlx-optiq
optiq serve --model mlx-community/LensVLM-9B-OptiQ-4bit

Images through the Python API:

from optiq.runtime.engine import OptiqEngine

eng = OptiqEngine("mlx-community/LensVLM-9B-OptiQ-4bit")
st = eng.generate("What is in this image?", images=["page.png"], max_tokens=256)
print(st.text)

Text-only works under stock mlx-lm as well — it globs model*.safetensors and ignores the sidecar, so the same repo loads either way:

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/LensVLM-9B-OptiQ-4bit")
print(generate(model, tokenizer, prompt="Explain KV caching.", max_tokens=200))

The per-layer bit map is in optiq/metadata.json and in the quantization block of config.json.

Links

• Project website: mlx-optiq.com
• PyPI: pypi.org/project/mlx-optiq
• Base model: apple/LensVLM-9B
• Paper: LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

License

Apple Machine Learning Research Model License (apple-amlr), inherited from the base model. Read the licence before use — it is not Apache 2.0, unlike the rest of the Qwen3.5 family.