experimentalmachines/LFM3-1.7B-Base-ExecuTorch

LFM3-1.7B-Base for ExecuTorch

ExecuTorch exports of LiquidAI/LFM3-1.7B-Base (revision 545fc8f4c7b5) for on-device inference with the openweights Android app or any ExecuTorch 1.5.1 runtime.

This is the base checkpoint, a completion model: it continues text and is not trained to follow chat turns or call tools. Give it a prefix to continue rather than a chat prompt.

The openweights app has no chat template for this model family yet, so it does not load these files until it gains one. Any ExecuTorch runtime with the LLM runner's custom ops runs them today.

ExecuTorch has no LFM3 definition of its own yet. These files are traced from ExecuPack's (pipeline/lfm3_model.py), which matches the checkpoint's own modeling_lfm3.py on fp32 logits, with ExecuTorch's custom KV-cache and SDPA ops: the windowed layers keep a ring of their last 256 tokens plus one prefill chunk instead of the full window.

Files

Windows in this repo: XNNPACK (CPU) at 2k to 32k. The window is fixed inside the file: the runtime allocates the whole KV cache at load, so pick the largest window the device can hold (fits_phone_budget in each folder's config.json is the estimate against a 5 GB budget).

Backend	Target	File	Window	Size	Smoke test
XNNPACK (CPU)	any arm64	xnnpack/LFM3-1.7B-Base-8da4w-gptq-2k.pte	2,048 tokens	1.24 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/LFM3-1.7B-Base-8da4w-gptq-4k.pte	4,096 tokens	1.24 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/LFM3-1.7B-Base-8da4w-gptq-8k.pte	8,192 tokens	1.25 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/LFM3-1.7B-Base-8da4w-gptq-16k.pte	16,384 tokens	1.25 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/LFM3-1.7B-Base-8da4w-gptq-32k.pte	32,768 tokens	1.27 GB	passed ("Paris")

Tokenizer: tokenizer.json, copied unchanged from the source repo. Each backend folder has a config.json listing every window as a variant with the metadata the .pte reports, and an export-report-<window>.json per file with the full export record.

Memory

• XNNPACK (CPU) at 2,048 tokens: the KV cache costs 24,576 bytes per token (fp32), 97,517,568 bytes for the whole window (47,185,920 of them the sliding-window layers' fixed ring), allocated in full when the model loads.
• XNNPACK (CPU) at 4,096 tokens: the KV cache costs 24,576 bytes per token (fp32), 147,849,216 bytes for the whole window (47,185,920 of them the sliding-window layers' fixed ring), allocated in full when the model loads.
• XNNPACK (CPU) at 8,192 tokens: the KV cache costs 24,576 bytes per token (fp32), 248,512,512 bytes for the whole window (47,185,920 of them the sliding-window layers' fixed ring), allocated in full when the model loads.
• XNNPACK (CPU) at 16,384 tokens: the KV cache costs 24,576 bytes per token (fp32), 449,839,104 bytes for the whole window (47,185,920 of them the sliding-window layers' fixed ring), allocated in full when the model loads.
• XNNPACK (CPU) at 32,768 tokens: the KV cache costs 24,576 bytes per token (fp32), 852,492,288 bytes for the whole window (47,185,920 of them the sliding-window layers' fixed ring), allocated in full when the model loads.

How it was made

• XNNPACK (CPU) any arm64: ExecuTorch 1.5.1 export_llm: 8-bit dynamic activations and 4-bit weights in groups of 32, int8 per-channel embeddings, XNNPACK with extended ops, prefill chunk 2048, fp32 KV cache.

License

A quantized derivative of LiquidAI/LFM3-1.7B-Base, distributed under the same terms (lfm1.0). The upstream license files are included unchanged: LICENSE.