Ovis2.5-2B — LiteRT-LM (on-device Vision-Language Model)

AIDC-AI/Ovis2.5-2B converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime (the engine behind the official litert-community/* models, and the same runtime that runs litert-community/FastVLM-0.5B).

Ovis2.5 is a SOTA-for-size vision-language model (OpenCompass ~73.9 for the 2B) with a distinctive structural-embedding vision path: a Siglip2 NaViT encoder feeds a visual tokenizer that turns each image patch-group into a probability distribution over a 65 536-word visual vocabulary, then embeds it — giving the language model image tokens that live in the same structured space as text. The language decoder is Qwen3-1.7B. This bundle runs the whole thing through LiteRT-LM's fast_vlm multimodal path — give it an image and a question, get a grounded answer, fully on-device.

File	Ovis2.5-2B.litertlm (~2.15 GB)
Vision	Siglip2-NaViT encoder + visual-tokenizer (head → softmax → visual-vocab embedding), int8 weights — single 512×512 image → 256 image tokens
Decoder	Qwen3-1.7B, int4 weights (symmetric, blockwise-32 + OCTAV optimal-clipping); input embedding INT8 (externalized section)
Compute	integer
Context (KV cache)	2048
Image input	resized to 512×512 (Siglip normalization is baked into the vision encoder)
Base model	AIDC-AI/Ovis2.5-2B (Apache-2.0)

Quality

The vision tower converts bit-faithfully to the reference — float CPU-parity end-to-end corr ≈ 1.0 (max abs diff 3e-6), with no FLEX/CUSTOM fallback ops; int8 vision weights keep end-to-end corr **0.99**. The Qwen3-1.7B decoder uses the same blockwise-32 + OCTAV int4 recipe that scores 90.7% GSM8K on the sibling Ministral-3-3B-Reasoning build and shipped the InternVL3.5-2B VLM. On a reference deployed-path eager run (fixed-512 vision → 256 tokens → Qwen3-1.7B) the model describes real photos accurately and in detail (e.g. a black-and-white Ansel-Adams-style landscape → "snow-capped sharp mountain peaks … a river winding through the valley … cloud layers … black-and-white contrast with depth of field").

Reasoning VLM. Ovis2.5 is a thinking model: it may emit a <think>…</think> block before its final answer (this matches the base model). Allow enough max-tokens (≥1024) for the answer to follow.

On-device performance: decode/load are expected to be in line with the InternVL3.5-2B build on the same runtime (~20 tok/s CPU, ~45 tok/s GPU on iPhone 17 Pro for single-image VQA). Independent on-device measurement for this specific build is recommended before quoting exact numbers.

Run on iPhone / macOS

Use the LiteRT-LM Swift runtime (swift-litert-lm / the LiteRTDemo sample). Load Ovis2.5-2B.litertlm with the image (vision) tower enabled (modalities [.vision] / Modality.textImage), attach a photo, and ask a question.

Note for app integrators: this is a vision-only bundle (no audio tower). Bring up the engine with the vision modality only — requesting the audio tower (.all) on a bundle with no audio section fails at session creation.

Run on Android — Google AI Edge Gallery

Install a recent Google AI Edge Gallery (1.0.16+ can import .litertlm directly from Hugging Face), download Ovis2.5-2B.litertlm, import it (tap +), attach an image and ask. The bundle already carries the tokenizer and prompt template.

Conversion notes

• LiteRT-LM fast_vlm bundle: VISION_ENCODER ([1,512,512,3]→[1,256,4608]) + VISION_ADAPTER ([1,256,4608]→[1,256,2048], matched to the Qwen3-1.7B hidden size) + single-token EMBEDDER + PREFILL_DECODE (embeddings-input).
• The NaViT static rewrite is the enabling trick. Ovis's Siglip2-NaViT vision tower is dynamic resolution (.item()/.tolist()/grid-loops/argsort) and does not torch.export. Because the config's fullatt_block_indexes=None makes every layer use full attention, the window-reorder is a mathematical no-op — so it can be dropped and replaced with a precomputed position embedding + rotary and a single full attention over the fixed 512×512 grid (1024 patches). Static-vs-original feature corr 0.99999964.
• The encoder bakes Siglip normalization ((x-0.5)/0.5, the runtime feeds a [0,1] NHWC image) and does patchify GPU-safe: the patch-embedding Conv2d is applied to the whole image (raster order), then a single gather reorders patches into Ovis's hidden-stride "merge" order — all reshapes ≤4D, no >5D op that GPU delegates reject.
• The adapter is Ovis's visual-tokenizer tail: head (Linear 4608→65532 + LayerNorm) → softmax → visual-vocabulary embedding (vte, 65536×2048). The 256-token bundle carries the visual atoms; Ovis's two learned image-boundary indicator embeddings are omitted (the fast_vlm path splices only the atom embeddings) — verified to stay coherent in eager.
• Decoder extracted from the Ovis2_5 wrapper as a standalone Qwen3ForCausalLM and exported with cache ≤ base max so base RoPE is exact.

License

Apache-2.0, inherited from the base model AIDC-AI/Ovis2.5-2B.