mlboydaisuke/LFM2.5-VL-1.6B-ExecuTorch

LFM2.5-VL-1.6B — ExecuTorch

lfm2_5_vl_1_6b_xnnpack_8da4w.pte (2.03 GB)

• Source: LiquidAI/LFM2.5-VL-1.6B — SigLIP 2 vision tower (1152 wide, 27 layers) and an LFM2 decoder (2048 wide, 16 layers)
• License: LFM Open License v1.0
• Input: a 512×512 tile as [1, 1024, 768] patches plus [1, 1024] attention mask, and token ids for the words around it
• Output: logits over the 64,400-token vocabulary

One file, three entry points — the shape ExecuTorch's multimodal runner asks for:

method	in	out
vision_encoder	patches, mask	[1, 256, 2048] rows in the decoder's embedding space
token_embeddings	token ids	embeddings
text_model	embeddings, positions	logits, and its own cache

A decoder that only takes token ids cannot be told about a picture. Splitting a vision-language model into a vision .pte and a text .pte runs aground there; MultimodalPrefiller::load asks one module for token_embeddings and text_model and uses vision_encoder if it finds it.

On a phone

1.64 GB, and its arena is 176 MB — it fits where the fp32 export (6.93 GB) did not. Not measured on device; the rate quoted for the 450M is that file's, not this one's.

Verification (Mac arm64, 2026-08-21)

Greedy, through the three methods, on photographs from convert/calib_images:

picture	answer
a London street	"A group of people are walking on a sidewalk in front of a building that says Pizza Express."
a man with a dog	"A black and white photograph shows a man standing next to a dog and a cart, with a large stack of wood in the background."

The shop sign is read correctly, which is the check that matters: a caption that fits any street would not tell you the vision half was wired up right.

The vision half of this file agrees with the untouched model at worst corr 0.97888 over 24 photographs, and the first-step logits at corr 0.97752 with the same top-1 token.

Square the picture first

The processor picks a tile grid from the aspect ratio — a 768×477 photograph becomes one 24×40 tile, a 1280×960 one becomes seven — and this graph takes 32×32. Centre-crop to a square before the processor sees it and every picture becomes exactly one tile. Feeding a stretched square instead is visible in the output: asked about a squashed street, the model called the scene "distorted and warped", which was a fair description of what it had been given.

python convert/run_vl.py <image> "What is in this picture?"

Conversion

convert/export_vl_bundle.py. Four things needed re-authoring:

• SigLIP 2 reads its grid out of a tensor to size the position embeddings, which torch.export cannot follow. The grid is fixed here, so the resize is computed once and the constant handed to a replacement forward.
• The projector wants the grid back. The tower returns a flat run of patches; the projector's pixel-unshuffle trades resolution for channels and needs to know which patches are neighbours.
• The quantization is not uniform. The decoder is int4 over groups of 32; the vision tower is int8 per output channel. At int4 throughout, the model reads the Pizza Express sign as "Pocket Express" — the letters live in the tower, and four bits does not hold them.
• The cache lives inside text_model. ExecuTorch copies a mutable buffer into each method that names it (Program::load_mutable_subsegment_into writes into the method's own memory), so a prefill method and a decode method would each get their own and neither would see the other's writes. One graph has to serve both, which rules out the two code paths transformers keeps for LFM2's short convolution — a windowed convolution for a prompt, a fused single-step update for a token, chosen in Python and therefore baked in by tracing. Carrying the last kernel - 1 columns and putting them in front of whatever arrives is the same arithmetic in one branch-free path.

The number that decides whether it runs on a phone

CONTEXT, the upper bound on the dynamic sequence dimension. The memory planner sizes its arena for the bound, not for what a picture costs: at 4096 that arena is 3.7 GB and iOS kills the process with signal 9 before the first method has finished loading. One square photograph is 1024 patches, which the projector unshuffles to 256 rows, so 512 leaves room for a long question and a long answer and brings the arena to 176 MB.

(conversion scripts: executorch-models · iOS sample: executorch-samples)

The token embedding table

The table is an nn.Embedding, and the quantisation filter in convert/export_vl_bundle.py was written as isinstance(module, nn.Linear), so it went out in fp32 — 537 MB of a 2031 MB file. tie_word_embeddings does not save it either: quantize_ swaps lm_head.weight for a new tensor and the embedding keeps pointing at the old fp32 one. (LFM2.5-VL ties and SmolVLM2 does not; both leaked.)

It is now int8 over groups of 64 along each row, written out by hand because torchao's IntxWeightOnlyConfig reaches an nn.Embedding but will not lower (Missing out variants: {'torchao::dequantize_affine'}). On this model it is free: the first-step logits read corr 0.97752 against 0.97812 for the fp32 table, the greedy token sequence is identical, and the file drops from 2031 MB to 1637 MB.

One scale per row is not enough for a narrow table: on SmolVLM2-256M, whose rows are 576 long, per-row scales moved the first-step logits from corr 0.99197 to 0.97811 and flipped the top-1 token, so that model keeps its fp32 table. Groups of 64 are what make this one free.