divinetribe/Vision-Narrator-0.8B-4bit-mlx

Vision Narrator 0.8B — 4-bit MLX

A vision-language model small enough to live on a phone, trained to say one useful sentence to someone who cannot see the screen.

This is the model that ships inside RealTime AI Cam. You point the camera at a room, or at a letter that came in the mail, and it tells you — out loud, offline, in about a fifth of a second — what is in front of you and what it means for you.

Not "a photo of a document." "This is a past due bill for $308.02, and they shut the service off on November 28th unless it's paid."

---

What makes it different

It is trained to be useful, not descriptive. A generic captioner tells a blind user "a piece of paper with writing on it." That is worthless. This model was trained to answer the four questions that actually matter on a letter: what is it, who sent it, how much, and what happens if you ignore it. On a scene, it answers what is in front of you and where.

The vision tower is not quantized. The language half runs at 4-bit (affine, group size 64). The part that actually sees is left alone. Crushing the encoder to save 40 MB is how you get a model that confidently misreads a dollar amount, and a wrong number spoken with confidence is worse for a blind user than no answer at all.

It is honest about failure. Part of the training set teaches it to say "I can't read this page, the photo is too dark. Add some light or turn on the flash and try again" instead of inventing a plausible bill. For this audience that behaviour is a feature, not a fallback.

It runs on a 2022 phone. Verified on an iPhone 14 Pro Max, fully offline, no network call at any point.

---

Real outputs

From the evaluation logs, unedited:

Scene — "A white mattress with a star pattern is right in front of you, so close it blocks most of the room. A person in purple pants is lying on a patterned rug to your left, with a bright orange heater sitting on the floor just ahead of them."

Bill — "This is a utility bill for $81.13, due October 8th, 2026. If it goes unpaid they can shut off service 15 days after that."

Nothing owed — "This is a Xfinity statement, and nothing is owed — the balance is zero."

Bad photo — "The top of the page is cut off, so I can't tell who sent this. Move the phone back a little so the whole page is in view and try again."

---

Architecture

Base	Qwen3.5 0.8B (Qwen3_5ForConditionalGeneration)
Language model	1024 hidden, 24 layers, 8 heads, 248,320 vocab
Vision tower	768 hidden, depth 12, projects to 1024
Quantization	4-bit affine, group size 64 — language model only, vision tower left at full precision
On disk	652 MB (596 MB of weights)
Runtime	MLX / MLX Swift

Training

LoRA, rank 16, scale 2.0, dropout 0, applied to all 24 language-model layers and nothing else — the vision tower was frozen throughout. The model was taught what to say about what it already sees, which is why a 0.8B encoder is enough.

Fine-tuned on a purpose-built set of rendered and camera-captured documents (utility bills, past-due notices, unclaimed-property letters, statements) plus indoor scenes, each paired with the one sentence a blind listener actually needs.

Evaluation

The harness scores three things and does not care how the prose sounds: facts (are the amounts, dates, senders and consequences right), short (one sentence, no preamble), and clean (no markdown, no hedging, nothing unspeakable by a screen reader).

Measured across the 0.8B line, August 2026:

Variant	Facts	Short	Clean	Avg
Fused bf16	366/400 — 91.5%	400/400	400/400	0.18s
4-bit language model	394/400 — 98.5%	400/400	400/400	0.17s
4-bit LM + 8-bit vision	385/400 — 96.2%	400/400	400/400	0.18s
Hard set (dark, cropped, skewed)	99/120 — 82.5%	120/120	120/120	0.18s

The hard set is deliberately unfair: photos too dark to read, pages with the header cut off, steep angles. 82.5% there is the number worth watching, because that is what a phone held by someone who cannot see the framing actually produces.

Use it

from mlx_vlm import load, generate
model, processor = load("divinetribe/Vision-Narrator-0.8B-4bit-mlx")
print(generate(model, processor, "photo.jpg", "What is this?", max_tokens=96))

On iOS it runs through MLX Swift — see RealTimeAICam.

Limits

English only. Tuned for US household documents and indoor scenes; a restaurant menu or a foreign utility bill is outside what it was taught. It will occasionally get a sender's name wrong on a badly lit page — it is a 0.8B model, not a human reader. Do not use it as the sole source of truth for anything financial or medical. It is a fast first look for someone who otherwise gets nothing.

Credits

Built on Qwen3.5 by Alibaba, quantized and served with MLX by Apple. Trained and released by divinetribe.