LFM2.5-2.6B for ExecuTorch, 32k context, CPU
LFM2.5-2.6B-8da4w-32k.pte: LFM2.5-2.6B compiled for the ExecuTorch 1.4.0 LLM runtime with the XNNPACK
backend, a 32,768-token context window, and weights quantised for Arm CPUs.
| File | 1815 MB |
| Context window | 32,768 tokens, fixed at export |
| Prefill chunk | 2048 tokens |
| Weights | int4, groups of 32, with int8 dynamic activations (8da4w); int8 embedding table |
| KV cache | fp32, allocated at load: about 1049 MB for the full window (8 of 30 layers attend (the rest are short convolutions)) |
| Resident memory after load | 2.94 GB, measured on a Dimensity 9400 |
| Speed on a Dimensity 9400 | about 149 tokens/s prefill, 18.1 tokens/s decode, awake and cooled, median of six turns on a 929-token prompt |
Why this quantisation
The export uses ExecuTorch's own recipe for the LFM2 family (examples/models/lfm2,
extension.llm.export.export_llm, XNNPACK partitioner). Int4 weights in groups of 32 with int8
dynamic activations is the layout Arm's KleidiAI micro-kernels take: on a CPU with the i8mm and
dotprod extensions, which every recent flagship and mid-range Arm phone has, XNNPACK dispatches
these matrix multiplies to KleidiAI, and that is where the prefill rate above comes from. There
is no GPU or NPU delegate in this file; it runs on the CPU everywhere.
What the window costs, and does not
Measured on four chips (Dimensity 9400, Snapdragon 8 Elite, Tensor G5, Exynos 2400) against exports of the same weights at 2k, 4k, 8k and 16k: the window changed memory at load and nothing else that could be told from run-to-run variation. Speed was within about 1% of the 2k export on a paired probe, because the runtime attends over filled positions only. Grades: GSM8K 26 of 30 and IFEval 21 of 30 at a 2048-token reply cap, BFCL 24 of 30 at a 640 cap, on the Dimensity 9400. The full study, with every table, is at https://alpharomercoma.github.io/openweights/window.html.
Give it room to reason
This model's chat template opens a thinking block before every answer, and it reasons for several hundred to two thousand tokens before it writes the reply. Two consequences:
- Set a reply budget of about 2048 tokens. At 640 it scored GSM8K 13 of 30 and IFEval 5 of 30 on the same prompts, not because it was wrong but because it was still thinking when the cap fell; at 2048 it scored 26 and 21.
- A window this size is the point: at a 2k window, 14 of 60 such replies ran into the window itself before finishing. At 4k and above none did.
Tokenizer and BOS
tokenizer.json is the base model's, with a post-processor added that prepends <|startoftext|> when a tokenizer is asked to add special tokens.
The ExecuTorch 1.4.0 LLM runner does not ask for special tokens, so an app must write
<|startoftext|> into the prompt text itself; the runtime's tokenizer encodes the literal to the
BOS id (124894). Without it this model misreads prompts and falls into repetition.
EOS is 124900. Chat template: the base model's ChatML-style template
(<|im_start|>role ... <|im_end|>).
Provenance
Weights are Liquid AI's, from LiquidAI/LFM2.5-2.6B, under the LFM Open License 1.0; this repository
only changes their packaging. Exported 2026-09-07 with ExecuTorch 1.4.0, torch 2.14, torchao 0.18
on an Apple M-series Mac in a few minutes; the recipe and the measurements are in the
OpenWeights repository under docs/research/.
Built for the OpenWeights Android app, which reads config.json to learn the window before
downloading.
- Downloads last month
- -