LFM3-VL-2B-MNN

Introduction

LFM3-VL-2B-MNN is an MNN-format export of LiquidAI/LFM3-VL-2B — a 2B-parameter language foundation model with native image understanding. The checkpoint was converted with llmexport and runs on the MNN runtime (llm engine + visual encoder) on Android, iOS, Linux and Windows, on CPU.

The text stack interleaves 13 short-convolution linear attention layers (attn_type: short_conv) with 11 full attention layers (sliding window 256, headwise sigmoid gate, fused RoPE, head_dim 128). Word embeddings are tied with the output projection, so the export keeps a single tied embedding table — no separate lm_head weights.

The vision side is a SigLIP-style ViT with a 2×2 patch-merge projector (hidden 4096, gelu, downsample_factor 2): a 1024×1024 frame becomes 64×64 patches of 16 px, merged to 1024 image tokens in the LLM sequence. The tower is grid-agnostic by design — positions come from axial 2-D RoPE computed from the patch grid (no absolute position-embedding table) and the upstream processor resizes dynamically between 0.25 MP and 16 MP; the 1024² square here is the static export size (≈1.05 MP, matching a native 720p frame), not a checkpoint limit. Export it at another size with LFM3_VL_EXPORT_SIZE if you need a different quality/latency point.

Quantization applied during export:

Part	Quant	Block	Notes
LLM linear weights	4-bit	64	HQQ calibration, alpha stored in fp16
Embedding / lm_head (tied)	4-bit	64	one shared table, --seperate_embed not used
Vision tower + projector	8-bit	128	visual.mnn.weight

Total download size is ≈ 1.19 GB (1,187,217,856 bytes).

Files

File	Size (bytes)	Content
llm.mnn	3,507,608	LLM graph (exported with conv4 cache off; decode verified ≈23 tok/s on an x86_64 CPU host)
llm.mnn.weight	973,847,872	LLM weights (4-bit, block 64)
visual.mnn	241,240	vision encoder graph — static packed-row graph (input rows [4096,1536] → image_embeds [1024,1,2048]); see "Vision graph fix" below
visual.mnn.weight	204,220,214	vision weights (8-bit, block 128)
tokenizer.mtok	5,386,284	tokenizer
llm_config.json	13,977	engine config: chat template, special tokens, vision_processor, image_size 1024
config.json	661	runtime defaults (backend, threads, max_new_tokens, precision)

Graph contracts: llm.mnn takes input_ids [1,1,2048], attention_mask [2,1,-1,-1], position_ids [1,-1] i32, logits_index [1] i32; visual.mnn takes packed merged patch rows [4096,1536] (static) → image_embeds [1024,1,2048] — 4096 rows = (1024/16)² patches, 1536 cols = C·T·p·p = 3·2·16·16 (2×2 merge is done by the engine's packPatches, not inside the graph).

Special tokens: bos 124894, eos 124900, image_token_id 125030.

Download

Method 1

Use pip to install the MNN package, then export any upstream checkpoint yourself:

pip install MNN

Method 2

Use the MNN command line tool to download this repository:

huggingface download --model Juxoola/LFM3-VL-2B-MNN --local_dir ./LFM3-VL-2B-MNN

Method 3

Use the Hugging Face Python API:

from huggingface_hub import snapshot_download
snapshot_download(local_dir="./LFM3-VL-2B-MNN",
                  repo_type="model",
                  repo_id="Juxoola/LFM3-VL-2B-MNN")

Usage

Build MNN with LLM and low-memory support, then run llm_demo against config.json:

cd MNN/build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true \
         -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true \
         -DMNN_BUILD_DIFFUSION=false -DMNN_OPENCV_BUILD=true
make -j
./llm_demo /path/to/LFM3-VL-2B-MNN/config.json prompt.txt

Use the CPU backend. OpenCL and Vulkan have no LinearAttention / fused RoPE kernels for this architecture, so GPU backends fail on the LFM3 layers. Image input requires the OpenCV-enabled build (MNN_SUPPORT_OPENCV / -DMNN_OPENCV_BUILD=true), because the runtime resizes every frame to the fixed 1024×1024 tensor the encoder expects.

Vision contract: input layout is pixels (vision_processor = {version: 2, family: "lfm3_vl", input_layout: "pixels"}), the frame is resized to 1024×1024 and expands to 1024 tokens (llm_config.json: image_size 1024, image_min_pixels = image_max_pixels = 1048576). In llm_config.json the model is declared with is_visual: true; image placeholders in the prompt are <img>image_0</img> markers.

Vision status (MNN 3.6.2, FIXED 2026-11-05). The stock llmexport of the lfm3_vl pair produced a dynamic vision graph (visual.mnn 402 552 B) that MNN's C++ engine silently mis-reshapes at runtime: the tower fed the LLM garbage embeddings and the model answered "I can't see images" (upstream export↔processor contract bug, reproduced with the stock llm_demo on x86_64). This repo ships a re-exported static vision graph (visual.mnn 241 240 B, traced with a fixed packed-row input [4096,1536]) so any wrong row count is a hard graph-input mismatch, not silent data reinterpretation. Vision is now proven working end-to-end on Android (see "OCR proof" below) and on the x86_64 host via llm_demo.

Engine requirements. The static graph only works with an MNN ≥ 3.6.2 engine whose Lfm3 VisionProcessor (in transformers/llm/engine/src/omni.cpp) has these two behaviors:

1. patchInputs: when llm_config.json advertises vision_processor.input_layout == "pixels", pack raw [T,3,H,W] pixels into merged rows (packPatches(..., "merged")) — the graph consumes packed rows, not raw pixels.
2. targetSize: always return the exported square (image_size, 1024×1024). A user-set vision_size override must be ignored for this family — e.g. 1120² → 4900 packed rows ≠ 4096 → the graph reshapes garbage and the model reverts to "I can't see images". This was the exact cause of the on-device failure before the engine patch.

Android .so requirement. libllm.so from an omni build dlopens libMNNAudio.so. Ship it in jniLibs/<abi>/ next to libllm.so, otherwise System.loadLibrary("mnnllm") fails with dlopen failed: library "libMNNAudio.so" not found and the LLM silently never loads.

The exported chat template is full Jinja with an enable_thinking switch (default true). With thinking on, the model opens a <think>…</think> block before the answer; with thinking off the template emits the closed empty block, and the answer follows directly. max_new_tokens in config.json is 2048, the context window is 8192.

OCR proof (device, this exact card)

Android ru.localassistant (MNN 3.6.2, CPU, x86_64 emulator) sent a 339 KB 720p test card with the prompt "Read ALL captions on this photo line by line, from the largest to the smallest text. Don't skip anything." The model decoded from the photo:

Сейт 720P
КОТ-4521
адрес: ул. Ленин, д. 7, кв. 42
пароль от владельца: COMERTY-720p-VISION
почта: info@comerty-720pp.com
регистрация   45-20

The 8-bit-quantized ViT garbles some words (Ленин vs the real street, COMERTY vs the real SHVONDERZ, Сейт vs СЕT), but the identifiers and numbers decode correctly: КОТ-4521, 720P, 720p-VISION, 7, 42, 45-20. This proves the full vision pipeline (photo → 512² → 1024 packed rows → ViT → 256 image tokens → LLM → text) works on-device. Full-quality OCR would need a bf16 vision tower.

Measured performance

CPU-only, 8 threads, LDPlayer Android 14 x86_64 on a desktop CPU (an Android app on MNN 3.6.2, not llm_demo):

• vision stage ≈ 3 s (the visual.mnn 1024×1536 forward on the emulator);
• full prefill of a photo turn (420 tok = 256 image + 164 text) ≈ 16–22 s;
• steady decode ≈ 9–12 token/s on photo turns;
• prefill ≈ 36–43 token/s, steady decode ≈ 8.9–9.2 token/s on 0.6–0.8 k-token text prompts;
• engine load + text warmup ≈ 10.5 s;
• same export on a desktop x86_64 host via llm_demo: vision ≈ 1.96 s, prefill ≈ 118.8 token/s, decode ≈ 22.9–24.2 token/s.

Document

• MNN LLM documentation
• llmexport guide
• Source model: LiquidAI/LFM3-VL-2B

This export is distributed under the LFM Open License v1.0 (see the source model card); it inherits the license and the usage restrictions of the base model.