experimentalmachines/LFM2.5-1.2B-Instruct-ExecuTorch-XNNPACK-32k

LFM2.5-1.2B-Instruct for ExecuTorch, 32k context, CPU

LFM2.5-1.2B-Instruct-8da4w-32k.pte: LFM2.5-1.2B-Instruct compiled for the ExecuTorch 1.4.0 LLM runtime with the XNNPACK backend, a 32,768-token context window, and weights quantised for Arm CPUs.

File	827 MB
Context window	32,768 tokens, fixed at export
Prefill chunk	2048 tokens
Weights	int4, groups of 32, with int8 dynamic activations (8da4w); int8 embedding table
KV cache	fp32, allocated at load: about 805 MB for the full window (6 of 16 layers attend (the rest are short convolutions))
Resident memory after load	1.77 GB, measured on a Dimensity 9400
Speed on a Dimensity 9400	about 343 tokens/s prefill, 40.5 tokens/s decode, awake and cooled, median of six turns on a 929-token prompt

Why this quantisation

The export uses ExecuTorch's own recipe for the LFM2 family (examples/models/lfm2, extension.llm.export.export_llm, XNNPACK partitioner). Int4 weights in groups of 32 with int8 dynamic activations is the layout Arm's KleidiAI micro-kernels take: on a CPU with the i8mm and dotprod extensions, which every recent flagship and mid-range Arm phone has, XNNPACK dispatches these matrix multiplies to KleidiAI, and that is where the prefill rate above comes from. There is no GPU or NPU delegate in this file; it runs on the CPU everywhere.

What the window costs, and does not

Measured on four chips (Dimensity 9400, Snapdragon 8 Elite, Tensor G5, Exynos 2400) against exports of the same weights at 2k, 4k, 8k and 16k: the window changed memory at load and nothing else that could be told from run-to-run variation. Speed was within about 1% of the 2k export on a paired probe, because the runtime attends over filled positions only. Grades: GSM8K 12 of 30, IFEval 18 of 30, BFCL 27 of 30 at a 640-token reply cap, on the Dimensity 9400; the same file run twice moved each grade by up to 3, so read those as plus or minus a few. The full study, with every table, is at https://alpharomercoma.github.io/openweights/window.html.

Tokenizer and BOS

tokenizer.json is the base model's. The ExecuTorch 1.4.0 LLM runner does not ask for special tokens, so an app must write <|startoftext|> into the prompt text itself; the runtime's tokenizer encodes the literal to the BOS id (1). The 1.2B tolerates a missing BOS, but the template expects it. EOS is 7. Chat template: the base model's ChatML-style template (<|im_start|>role ... <|im_end|>).

Provenance

Weights are Liquid AI's, from LiquidAI/LFM2.5-1.2B-Instruct, under the LFM Open License 1.0; this repository only changes their packaging. Exported 2026-09-07 with ExecuTorch 1.4.0, torch 2.14, torchao 0.18 on an Apple M-series Mac in a few minutes; the recipe and the measurements are in the OpenWeights repository under docs/research/. Built for the OpenWeights Android app, which reads config.json to learn the window before downloading.