alpharomercoma/LFM2.5-2.6B-ExecuTorch-XNNPACK

LFM2.5-2.6B for ExecuTorch (XNNPACK), several context windows

The same weights exported 5 times, differing only in the context window compiled into the static KV cache. Exported with ExecuTorch 1.4.0's own recipe for the family (examples/models/lfm2, extension.llm.export.export_llm): int8 dynamic activations, int4 weights in groups of 32 (the layout Arm KleidiAI kernels take in the XNNPACK runtime), int8 embedding table, prefill chunk 2048 tokens. Runs in any app on the ExecuTorch 1.4.0 Android or iOS LLM runtime; built for OpenWeights.

File	Window (tokens)	Size	KV cache at load (fp32)
lfm2_5_2_6b_8da4w_2k.pte	2048	1783 MB	about 66 MB
lfm2_5_2_6b_8da4w_4k.pte	4096	1786 MB	about 131 MB
lfm2_5_2_6b_8da4w_8k.pte	8192	1790 MB	about 262 MB
lfm2_5_2_6b_8da4w_16k.pte	16384	1798 MB	about 525 MB
lfm2_5_2_6b_8da4w_32k.pte	32768	1815 MB	about 1050 MB

Pick by memory, not speed: the runtime attends over filled positions only, so a larger window costs memory at load and nothing per token (measured: every window within about 1% of the smallest on prefill and decode in a paired probe). One exception to "nothing but memory": this model reasons before it answers, and a reply of that kind runs to 1000 to 2000 tokens. At a 2k window 14 of 60 such replies hit the edge of the window itself and were cut; at 4k and above none did. Give it a reply budget of about 2048 tokens and a window of 4k or more (measured GSM8K 23 to 28/30 and IFEval 18 to 21/30 on four chips under that budget, against 13 to 19 and 3 to 6 at 640). Measured on a Dimensity 9400 (Poco X8 Pro), awake, median of six turns: prefill about 154 tok/s, decode about 20 tok/s, 2.40 GB resident at 16k. The full window matrix on four chips is in the OpenWeights repository under docs/research/window-matrix.md.

tokenizer.json is the base model's, with a post-processor added that prepends <|startoftext|> when a tokenizer is asked to add special tokens. Note that the ExecuTorch 1.4.0 LLM runner never asks: an app must write <|startoftext|> into the prompt text itself (the C++ tokenizer encodes the literal to the BOS id). Without BOS this model answers garbage; the 1.2B tolerates its absence. BOS 124894, EOS 124900. Chat template: the base model's ChatML-style template (<|im_start|>role ... <|im_end|>).

Weights are Liquid AI's, under the LFM Open License 1.0; this repository only changes their packaging.