Mage-VL — LiteRT-LM (on-device Vision-Language Model)

microsoft/Mage-VL converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime — the first Mage-VL in this format.

Mage-VL is Microsoft's 4.7B vision-language model: a 24-layer ViT with 3-D rotary position embeddings feeds a Qwen3-4B language decoder. It is a strong general describer and a very good document reader for its size. This bundle runs the image path through LiteRT-LM's fast_vlm runtime — give it an image and a question, get a grounded answer, fully on-device.

File	Mage-VL.litertlm (~2.81 GB)
Vision	mage_vl_vision ViT (24L, 1024-dim, full attention, 3-D rope) made static 448×448 → 784 patches → 2×2 merge → 196 image tokens, int8 weights
Adapter	PatchMerger (LN → 2×2 group → MLP), int8, output at the 2560 text hidden size
Decoder	Qwen3-4B (36L, hidden 2560, GQA kv8), int4 weights (symmetric, blockwise-128 + OCTAV); int8 externalized embedder
Context (KV cache)	2048
Image input	resized to 448×448 (OpenAI-CLIP normalization baked into the encoder)
Base model	microsoft/Mage-VL (Apache-2.0)

Performance (measured)

Platform	Decode	Time-to-first-token (image turn)	Engine init	Peak footprint
iPhone 17 Pro (CPU, release build)	~10 tok/s	~7.5–9 s (includes vision encode + prefill)	0.8 s warm / ~3–4 s first run	~1.5 GiB
macOS (Apple Silicon, LiteRT-LM CPU)	~66 tok/s	~2 s	~1 s	—

Text-only follow-up turns in the same conversation have ~2 s time-to-first-token on the phone.

Quality

Device-verified on an iPhone 17 Pro and on the desktop LiteRT-LM runtime (macOS CPU):

• General description / VQA (photo, on-device): accurate, detailed, and identical to the desktop runtime output token-for-token — an Ansel-Adams-style landscape → "a black and white photograph of a mountainous landscape … a winding river cutting through a dense forested valley … jagged, snow-capped mountains … heavy, brooding clouds".
• Document OCR (Extract all the text from this image., on-device): perfect transcription of a full synthetic report page — every figure, the e-mail address and the phone number, at 448×448.
• Vision tower: static-rewrite vs the reference implementation corr 1.0 (fp32), 0.994 at int8, zero FLEX/CUSTOM ops; patch pipeline verified bit-identical to the model's own image processor (max diff 2.4e-7).
• Decoder: the Qwen3-4B text model is re-hosted as a standalone Qwen3ForCausalLM (state-dict strictly 1:1, untied lm_head) and quantized with the blockwise-128 + OCTAV int4 recipe that the shipped Qwen3-4B-class LLMs use; desktop 8-question sanity gate 7/8 with no degeneration.

No positional compromise. Unlike Qwen2-VL-family bundles, Mage-VL's language decoder natively uses plain sequential 1-D positions (no M-RoPE), which is exactly what the fast_vlm runtime supplies — the deployed decoder contract is mathematically identical to the original model. The 2-D-table-ranking caveat of the Qwen2-VL bundle does not apply here.

Image-only bundle. The base model's video pipeline (neural-codec frame compression) is not included — this bundle handles single images. Send each image in its own message; multiple images in one conversation are untested.

Run on iPhone / macOS

Use the LiteRT-LM Swift runtime (swift-litert-lm). Load Mage-VL.litertlm with the vision tower enabled (Modality.textImage), attach a photo, and ask a question.

Vision-only bundle (no audio tower): bring the engine up with the vision modality only — requesting .all fails at session creation on bundles without an audio section.

Run on Android — Google AI Edge Gallery

Install a recent Google AI Edge Gallery, download Mage-VL.litertlm, import it (tap +, enable "Support image"), attach an image and ask. (Verified platforms above are iPhone and macOS; Gallery import follows the same bundle contract as the other fast_vlm models here.)

Conversion notes

• LiteRT-LM fast_vlm bundle: VISION_ENCODER ([1,448,448,3]→[1,784,1024]) + VISION_ADAPTER ([1,784,1024]→[1,196,2560]) + single-token EMBEDDER + PREFILL_DECODE (embeddings-input), ChatML prompt with <|vision_start|>…<|vision_end|> image markers.
• Static rewrite of the dynamic-res vision tower. Mage-VL's ViT is native-resolution (packed patches, grid_thw, cu_seqlens varlen attention) and does not torch.export. The static graph fixes 448×448; a single image is one attention chunk, so the varlen machinery reduces to plain full attention.
• Direct Conv2d patchify. temporal_patch_size=1, so the patch-embed is a stride-16 Conv2d — applied to the whole image in raster order (no per-patch reshuffle, no Conv3d fold needed).
• 3-D rope as a constant. The tower's rotary embedding splits head_dim 4:6:6 over (t,h,w) with interleaved rotation; for a single image t=0, and the (h,w) frequencies are precomputed from raster patch positions and baked into the graph.
• No GATHER_ND (mobile-GPU-safe graph). Patches stay in raster order through the (permutation-equivariant) encoder; the 2×2 merge happens in the adapter with 4 strided slices + concat, all ops ≤4D. Static-rewrite corr vs the reference stays 1.0.
• Context is 2048 (vs the trained 262k): the right on-device trade for a 4B decoder — the fp32 KV cache stays ~0.6 GB, keeping the whole session under ~1.5 GiB on the phone.

License

Apache-2.0, inherited from the base model microsoft/Mage-VL.