litert-community/LFM3-VL-2B-LiteRT

LFM3-VL-2B — LiteRT-LM

This repository is a conversion of LiquidAI/LFM3-VL-2B to LiteRT-LM, made with litert-torch (release 0.10.0 for the vision encoder, commit f61d0b7 of its main branch for the text decoder) and litert-lm-builder 0.18.1 for LiteRT-LM 0.18.1: on a Galaxy S26 GPU it decodes 14.72 tokens/s after a 1,024-token prompt (median of three runs, 2026-10-29).

LFM3-VL-2B is Liquid AI's vision-language model for on-device visual understanding: 1.92B parameters, a 1.73B LFM3 language model (24 layers: 13 gated short convolutions, 5 sliding-window attention, 6 global attention), a 164M vision encoder (14 layers: 8 gated depthwise 2D convolutions, 6 bidirectional attention) and a 25M projector. It reads text, images and video, and by default it thinks in a <think> block before it answers. This file takes text and images; video is not included.

Files

File	What is inside	Size	sha256
LFM3-VL-2B_f16.litertlm	Text decoder with fp16 weights (no activation type declared), token embedding table in int8, vision encoder with fp32 weights and adapter with fp16 weights (both declared fp32 activations). One image becomes 256 tokens at 512 × 512. Context 2,048 tokens.	4,439,967,995 B (4.44 GB)	e9e15f4da346a8eb7fc5d2dc276dde9a9cecf1424d50193deeb2d098bf664877
LICENSE	The source model's license file, unchanged	10,574 B
litertlm_manifest.json	Machine-readable description of the .litertlm file	13,600 B
fixtures/	The four 512 × 512 images of the demo below	0.9 MB

The source repository also lists Liquid AI's own GGUF, MLX and ONNX checkpoints.

Usage

Turn the photo upright and resize it to 512 × 512 first. LiteRT-LM 0.18.1 stretches every image to the file's 512 × 512 (it keeps neither the aspect ratio nor the EXIF orientation), and every result on this page was measured on 512 × 512 images made with this line (the NASA photos were first reduced to 1,024 pixels on the long side):

pip install pillow
python -c "from PIL import Image, ImageOps; ImageOps.exif_transpose(Image.open('photo.jpg')).convert('RGB').resize((512, 512), Image.BICUBIC).save('photo_512.png')"

One question about an image with the LiteRT-LM command-line tool (thinking off):

uvx --from litert-lm==0.18.1 litert-lm run LFM3-VL-2B_f16.litertlm --backend gpu --vision-backend gpu --cache no --thinking false --attachment photo_512.png --prompt "What words are written on the yellow sign? Answer with the words only." < /dev/null

Without --thinking false the tool prints the model's thought between [thought] and [/thought] before the answer.

The same from Python (pip install litert-lm==0.18.1):

import litert_lm
from litert_lm import interfaces

engine = litert_lm.Engine("LFM3-VL-2B_f16.litertlm",
                          backend=interfaces.Backend.GPU(),
                          vision_backend=interfaces.Backend.GPU(),
                          cache_dir=":nocache")
conversation = engine.create_conversation(extra_context={"enable_thinking": False})
reply = conversation.send_message(
    {"role": "user", "content": [
        {"type": "image", "path": "photo_512.png"},
        {"type": "text", "text": "What words are written on the yellow sign? Answer with the words only."}]},
    max_output_tokens=64)
print("".join(part["text"] for part in reply["content"] if part.get("type") == "text"))

Thinking is on by default, as in the source model. Leave out extra_context (or pass {"enable_thinking": True}) to turn it on; the thought then comes back in reply["channels"]["thought"] and the answer in reply["content"]. Give it room (the example above stops at 64 tokens with thinking off; with thinking on use, for example, max_output_tokens=512): on our 32 test prompts the longest thinking answer was 352 tokens (Limitations). The file carries no sampler settings, so answers are greedy unless you set a sampler; the source model card recommends temperature 0.1, top_k 50 and repetition_penalty 1.1 (LiteRT-LM 0.18.1's sampler settings have no repetition penalty). Pass interfaces.Backend.CPU() for both backends to run on the CPU.

On Android, litert_lm_advanced_main built from the LiteRT-LM v0.18.1 tag reads the image from the prompt: a prompt file holding [image:/data/local/tmp/photo_512.png]What words are written on the yellow sign? Answer with the words only. and --backend=gpu --vision_backend=gpu --model_path=LFM3-VL-2B_f16.litertlm --input_prompt_file=<that file> --max_num_tokens=2048. This command-line tool has no thinking switch: our phone runs used a copy of this file whose template defaults to thinking off (12 bytes differ; every other byte is the same). With this file as published, the tool prints the model's thought before the answer. On the Galaxy S26 the GPU engine took 28–39 s to start in our runs.

Performance

Galaxy S26 (SM-S942Q, Android 16), litert_lm_advanced_main built from the LiteRT-LM v0.18.1 tag.

Backend	Prefill, 1,024-token prompt (tok/s)	Decode, 256 tokens (tok/s)	Time to first token, 1,024-token prompt (s)	Image + question (~280 tokens): first token (s)	Peak memory (MB)
GPU, vision encoder on the GPU	1,236	14.72	0.90	1.12 ¹	4,431–4,589
GPU, vision encoder on the CPU				1.53 ²	4,557–4,589
CPU, 4 threads	74.31	5.28	13.97	25.4 ³	7,642

• NPU: this file is not built for the NPU. A separate build compiled ahead of time for this phone's NPU (Qualcomm QAIRT 2.47) is not published: its answers did not match the source model's first 32 tokens on 1 of the 8 test pictures it was checked on (the same words in capitals, "OVERSIZE LOAD").
• Text rows: --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048, no vision encoder loaded. GPU: median of three processes (decode 13.63, 14.72 and 15.35 tok/s). In those runs the phone held its CPU clock below the maximum in 69 of 70 samples taken every 2 s (the faster cores at 37 % of their maximum at the lowest), and during decode it lowered the GPU's clock limit from 1,300 MHz to as low as 500 MHz. CPU: one process; the CPU clock was held below the maximum in every sample.
• Image rows: a 512 × 512 image and a question, 276–290 prompt tokens (256 for the image). First token = the runtime's vision encoder time plus its time to first token. ¹ Median of the 23 questions of the accuracy run below (one process per question, run back to back): vision encoder 375 ms, then 0.74 s. ² The truck photo of the demo below, median of three processes: median of the three per-process sums (vision encoder 971 ms on the CPU and 0.57 s after it, as medians). ³ The same photo, one process, the run that wrote the CPU weight cache: vision encoder 559 ms, then 24.81 s.
• Peak memory is the runtime's own "Peak system ram usage" line. On the CPU, XNNPACK unpacks the fp16 decoder into a 6.92 GB float32 weight cache next to the model.

Pixel 8a (Mali-G715, Android 16), the same litert_lm_advanced_main build, GPU with the vision encoder on the CPU: during the accuracy run below (8 images, one process per question, run back to back) the first token came 11.4 s after the request (vision encoder 7.2 s, then 4.15 s), prefill ran at 85 tok/s and the engine took 78 s to start. Our test stops a run at the first reading of available memory under 400,000 kB (read every 0.25 s on the phone); the lowest reading in these 16 runs was 401,624 kB, so the file runs on this phone with almost nothing to spare. The Pixel 8a CPU was not tried: the CPU path needs the 6.92 GB weight cache above, and this phone had 4,294,304–4,701,716 kB (4.40–4.81 GB) available before each run.

Mac M4 Max (macOS 27.0), litert-lm 0.18.1 from PyPI (its GPU path is WebGPU), Python API, cache_dir=":nocache", one process per run, median of three. Other jobs ran on the same Mac (load average 6–17).

Backend	Prefill 1,024 (tok/s)	Decode 256 (tok/s)	Time to first token (s)	Image + question: first text (s)	Peak footprint text / image (GB)
GPU (vision encoder on the GPU)	3,932	99.0	0.271	0.220	5.04 / 5.92
CPU (vision encoder on the CPU)	557	17.3	1.90	3.93	43.84 / 44.54

The image column uses the truck photo and question of the demo below; first text = vision encoder + prefill + first decoded token, as streamed.

On-device demo

Q: What words are written on the yellow sign? Answer with the words only. A: "Oversize load"	Q: Describe this image in one sentence. A: "Three men wearing orange jumpsuits and hoods are working together in a forest, handling large green fern-like plants."
Q: Which day has the tallest bar in this chart? Answer with the day label only. A: "Wed"	Q: What text is written on the sign? Answer with the text only. A: "EXIT 12"

Verbatim answers, thinking off, greedy, litert_lm_advanced_main built from the LiteRT-LM v0.18.1 tag: the Galaxy S26 (--backend=gpu --vision_backend=gpu) and the Pixel 8a (--backend=gpu --vision_backend=cpu) gave the same four answers, word for word the Mac GPU's. With the vision encoder on the Galaxy S26's CPU the sign came back as "OVERSIZE LOAD" (three runs out of three). The two photos are NASA's (public domain): KSC-08pd3809 and S67-34267; the chart and the sign are our own drawings.

Accuracy

The test set is 23 questions about 12 images (8 drawings and 4 NASA photos): 18 with one right answer (counts, colours, text, chart and table values, left or right, yes or no), 3 bounding boxes and 2 one-sentence descriptions; plus 8 short text questions and "What is the capital of France?". The reference is the source model in Transformers on the CPU, given the same 512 × 512 images. All answers here are greedy, one process per question.

• Mac GPU: on all 23 image questions the first 32 tokens of the answer equal the source model's in float32, with thinking off and with thinking on (46 of 46), and the text answers equal float32's (8 of 8). Mac CPU: 23 of 23 thinking off, 22 of 23 thinking on, text 8 of 8.
• The 18 questions with a right answer: 18 correct on the Mac GPU, the Mac CPU and the Galaxy S26 (the Pixel 8a: the 7 asked). The text questions: 7 of 8 pass on the Mac and the Galaxy S26 (the Pixel 8a: the 7 asked pass), and France is "Paris" everywhere; the miss is a rhyme the source model misses too.
• Against the source model in bfloat16 (its released precision) the first 32 tokens agree on 22 of 23 thinking off and 20 of 23 thinking on (Mac GPU; Mac CPU 22 and 19). bfloat16 and float32 themselves differ on 1 and 3 of these questions.
• Galaxy S26 GPU against the Mac GPU: 22 of 23 image answers identical; the 23rd is a bounding box with two of its four coordinates one unit off. The text answers are identical (9 of 9). Pixel 8a GPU: the 8 image answers asked are identical, and so are the 7 text questions asked (the eighth was not reached) and France.
• Images at a fixed 512 × 512 instead of the source processor's native resolution (which keeps the aspect ratio; 256–800 image tokens on these images): with the source model in float32, all 18 right answers stay right, and the first 32 tokens stay the same on 16 of 23 questions (boxes move a few units, descriptions are reworded).

Limitations

• Every image is stretched to 512 × 512 = 256 tokens. Turn photos upright and resize them yourself (Usage).
• Video is not in this file. More than one image per conversation was not tried.
• Context is 2,048 tokens (the source model: 128K). With thinking on, every answer to the 32 test prompts ended within 634 tokens, prompt included (measured on a 4,096-token build of the same weights, whose replies to all 64 test prompts equal this file's).
• Create one Engine per conversation: on LiteRT-LM, a conversation started on an Engine that already served another one continues from that conversation's running state (LiteRT-LM issue #3165). This model keeps a running short-convolution state; this was not measured on this file.
• From the second message of a conversation, LiteRT-LM 0.18.1 feeds the previous answer's stop token to the model as a zero embedding (every file with a separate embedding table); not measured on this file. One question per conversation avoids it.
• Pixel 8a: the lowest free-memory reading was 1,624 kB above our stop line (Performance).
• The Galaxy S26 CPU: 5.28 tok/s, with a 6.92 GB weight cache. Use the GPU.
• Tool calling was not tested with this file.
• The source model card does not recommend the model for coding, multi-turn conversations or complex reasoning.

Conversion notes

• Source: LiquidAI/LFM3-VL-2B at revision f83d2b0a (1,918,181,888 BF16 parameters). The current main (da10385c) changes only README.md.
• Text decoder: exported with litert-torch (commit f61d0b7) and our LFM3 export extension (see REPRODUCE.md in the conversion repository) (torch 2.14.0, transformers 5.20.2) with 7 signatures (prefill 1,024 / 256 / 64 / 16 / 4 / 1 and decode) over a 2,048-token cache, the token embedding table as its own section; every fully connected weight cast to fp16 with ai-edge-quantizer, the embedding table in int8. Before any converted file was measured, we required at least 21 of the 23 image answers (thinking off) to equal the bfloat16 source model's first 32 tokens. Int8 and int4 decoder weights, whole or in part, gave 17–20; a decoder with int8 weights in its first 8 layers and in every short-convolution projection passed on the Mac but changed one of 8 answers on the Pixel 8a GPU. So the decoder ships with fp16 weights.
• Vision encoder and projector: re-written in plain PyTorch for the fixed 512 × 512 grid and exported with litert-torch 0.10.0. The graph takes the runtime's float(u8) / 255 input back to u8 and normalises it as the source processor does. Against the source model's image features the worst per-token cosine over 12 images is 0.9999978 for this fp32 encoder and fp16 adapter; fp16 or int8 encoder weights, or the GPU's fp16 precision, lowered it to 0.20–0.70, so the encoder stays fp32.
• Template: the source chat template with the changes LiteRT-LM's template engine needs (no generation tags, no .get() calls) and the image placeholder. On 10 test conversations (images, text, a system message, two turns with a thought, two images in two turns) it renders the token ids of the source processor, and LiteRT-LM 0.18.1's own rendering equals ours on 5 conversations (an image with thinking off, on and unset; a second turn with thinking off and on). The tokenizer is the source repository's tokenizer.json, byte for byte.

License

LFM Open License v1.0, the same license as the source model (LICENSE). The files here are converted from LiquidAI/LFM3-VL-2B: the weights are cast to fp16 (text decoder, vision adapter) and int8 (token embedding table), the vision encoder is re-exported in fp32, and the chat template is changed as described above. Section 5 of the license does not license commercial use by an organization with annual revenue of 10 million US dollars or more (a qualified non-profit's non-commercial and research use is exempt).