alpharomercoma/LFM2.5-1.2B-Instruct-ExecuTorch-XNNPACK

LFM2.5-1.2B-Instruct for ExecuTorch (XNNPACK), several context windows

The same weights exported 5 times, differing only in the context window compiled into the static KV cache. Exported with ExecuTorch 1.4.0's own recipe for the family (examples/models/lfm2, extension.llm.export.export_llm): int8 dynamic activations, int4 weights in groups of 32 (the layout Arm KleidiAI kernels take in the XNNPACK runtime), int8 embedding table, prefill chunk 2048 tokens. Runs in any app on the ExecuTorch 1.4.0 Android or iOS LLM runtime; built for OpenWeights.

File	Window (tokens)	Size	KV cache at load (fp32)
lfm2_5_1_2b_8da4w_2k.pte	2048	796 MB	about 49 MB
lfm2_5_1_2b_8da4w_4k.pte	4096	798 MB	about 98 MB
lfm2_5_1_2b_8da4w_8k.pte	8192	802 MB	about 197 MB
lfm2_5_1_2b_8da4w_16k.pte	16384	810 MB	about 394 MB
lfm2_5_1_2b_8da4w_32k.pte	32768	827 MB	about 787 MB

Pick by memory, not speed: the runtime attends over filled positions only, so a larger window costs memory at load and nothing per token (measured: every window within about 1% of the smallest on prefill and decode in a paired probe). Measured on a Dimensity 9400 (Poco X8 Pro), awake, median of six turns: prefill about 360 tok/s, decode about 43 tok/s, 1.38 GB resident at 16k. The full window matrix on four chips is in the OpenWeights repository under docs/research/window-matrix.md.

tokenizer.json is the base model's. BOS 1, EOS 7. Chat template: the base model's ChatML-style template (<|im_start|>role ... <|im_end|>).

Weights are Liquid AI's, under the LFM Open License 1.0; this repository only changes their packaging.