visible-cx/LFM2.5-8B-A1B-CoreAI

LFM2.5-8B-A1B — Core AI (.aimodel)

LiquidAI/LFM2.5-8B-A1B as a Core AI .aimodel bundle for Apple silicon, published by visible-cx. mf64/ is a re-export that adds the S=64 chunked-prefill entrypoint and a terminated reasoning-block chat template; the defective upstream mirror it replaces has been removed from this repo.

It is a sparse mixture-of-experts model: 8.3B total parameters, ~1.5B active per token. 24 layers — 18 short-convolution mixers and 6 GQA attention layers; the first two layers are dense, every layer after is a 32-expert top-4 MoE. The expert weights are read through a gather_qmm Metal kernel, which is what makes the model fast: upstream measured 140.4 tok/s on an M4 Max through the gather path against 39.2 tok/s for the same weights read as a dense over-read.

Provenance

Base checkpoint	LiquidAI/LFM2.5-8B-A1B
Zoo recipe	models/lfm2.5-8b-a1b-moe/recipe.toml → ["lfm2.5-8b-a1b"], status = "verified"
Recipe	export_lfm2_moe_multifunction.py — metalize_moe_batched installs BatchedMetalSwitchGLU
Producer fingerprint	coreai-core 1.0.0b2 on every inner asset metadata.json
Asset creation date	20260817T231316Z
Weight format	sym8 — symmetric-linear int8, per-K-block-32 scale, read through the gather_qmm Metal kernel
Vocab	128,000
Export functions	main (S=1 decode) + prefill (S=64)

sym8, not int4. An int4km-gather variant exists upstream and is deliberately not published: non-QAT int4 does not hold quality on this model (+12 output flips out of 41 against the fp16 reference), where sym8 is clean (+1 flip out of 41, the fp16 ceiling).

Non-expert weights are the shipped int8 per-block-32 recipe calibrated on the decode spec. BatchedMetalSwitchGLU serves S=64 by sorting (token, expert) pairs and falls back to the q=1 gather path at S=1, so both entrypoints keep the gather kernel.

Contents

Path	Bytes	Files	Manifest context	Functions
mf64/lfm2_5_8b_a1b_decode_sym8_gather_mf64	9,447,767,758	7	16384	main + prefill

The folder holds <name>.aimodel/ (main.mlirb ≈ 9.43 GB, main.hash, asset metadata.json), a bundle-level metadata.json, and tokenizer/ (tokenizer.json, tokenizer_config.json, chat_template.jinja).

Stop token: eos_token = "<|im_end|>".

Chat template

LFM2.5-8B-A1B is an always-thinking model, so the generation prompt terminates the reasoning block — {{- "<|im_start|>assistant\n<think></think>\n" -}}, the same form the LFM2.5-2.6B bundles use. Free-form generation therefore returns text with the bundle exactly as published (4,561 characters on the 3.4k measurement below) rather than spending the whole budget inside an unterminated <think> block. Guided decoding is unaffected either way, because the grammar forces structured output from the first token.

Requirements

• Apple silicon Mac, Core AI runtime. Mac-only — a 9.4 GB int8 bundle does not fit an iPhone even with the memory-limit entitlement.
• Engine contract: 2 inputs (input_ids, position_ids) → logits. No static inputs, no per-step mask.
• Memory, measured: max RSS 9.0–10.7 GB, peak in-process footprint 0.16–0.39 GB. Minimum practical machine memory: 16 GB, with little else resident — the model runs on that tier but leaves almost no headroom. 24 GB or more is the comfortable tier.
• Disk: the 9.4 GB bundle plus roughly 17 GB of graph-compile cache on first load.
• The bundle manifest declares runtime_env COREAI_CHUNK_THRESHOLD = "1", the most memory-conservative prefill setting available. Keep it.

Performance

Measured on a 16 GB Apple silicon Mac (M2 Pro, macOS 27 beta), sequential engine: 10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, reset() between samples. Load excluded from s/row; sample 1 excluded as a cache-warm outlier.

Chat template unmodified.

	mf64/
Guided JSON parse	10/10
Enum-clean	10/10
s/row (long samples)	7.70
s/row (short samples)	4.71
Decode, guided	51.9–53.5 tok/s
TTFT, guided	1.8–13.8 s
Prefill	6.3 ms/token
Peak footprint	0.39 GB
Max RSS	9.04 GB

Unguided, 900-token cap:

depth	prompt tokens	TTFT	decode	generated	wall	chars	max RSS
3.4k	3,249	22.34 s	50.97 tok/s	821	38.4 s	4,561	10.73 GB
8k	7,673	48.07 s	47.67 tok/s	900	66.8 s	5,084	10.55 GB

Needle-in-haystack: 3/3 at 8k and 3/3 at 15k. Three distinctive facts planted at 10/50/90 % of the filler, strict scoring. 8k: 7,813 prompt tokens, TTFT 49.60 s. 15k: 14,566 prompt tokens, TTFT 91.41 s, 44.63 tok/s, peak footprint 0.39 GB — reachable because the manifest declares 16384 and the prefill entrypoint makes that depth practical. Per-prompt-token prefill cost is the same 6.3 ms at 8k and 15k, so it is linear in depth.

Cold-load caveat. The first load of a freshly downloaded copy was 60.6 s, since it includes the graph compile. Subsequent loads were 22.9–26.2 s. Budget roughly 17 GB of on-disk compile cache in addition to the 9.4 GB bundle.

Decode is healthy for the size — an MoE with ~1B active parameters decoding faster than a dense 2.6B.

Status

Artifact	Status
mf64/lfm2_5_8b_a1b_decode_sym8_gather_mf64	QUALIFIED. Measured: 10/10 guided parse and enum-clean, 51.9–53.5 tok/s decode, 7.70/4.71 s/row, 3/3 needle recall at both 8k and 15k, and non-empty free-form output with the chat template unmodified. Manifest context 16384.

License

LiquidAI/LFM2.5-8B-A1B is released under the LFM Open License v1.0 (lfm1.0). These bundles are a derivative of that checkpoint and the same licence and obligations travel with them — see the upstream licence. Redistribute the licence with the files.

mf64-tp — token-tiled prefill kernel

RETIRED — failed its Mac gate on 2026-08-18. Kept for the record only. Do not deploy it and do not route traffic to it. Production use: mf64/.

RETIRED - DO NOT DEPLOY: measured on Apple silicon 2026-08-18 and it failed two of its three headline claims. (1) The Metal kernel DOES compile and run - that gate passed on the first attempt. (2) It is NOT bit-identical: 3 of 10 guided rows diverge from mf64, including one that invents an entity absent from the mf64 answer and one that flips a schema enum. A same-session mf64 re-run reproduced its own recorded outputs byte-for-byte on all 10 rows, so the harness is deterministic and the divergence is the kernel. Note both bundles parse 10/10 and are enum-clean, so a schema-shape gate would NOT have caught this. (3) Prefill is 5.35 ms/prompt-token at 8k against a prediction of 0.97 (range 0.85-1.39) - a 1.17x speedup, not the predicted 6.5x; 8k TTFT 48.07 s -> 41.03 s, not the predicted ~7.4 s. Short-prompt enrichment REGRESSED to 8.66/5.74 s/row against a same-session mf64 re-run at 6.86/4.08. Decode is genuinely unchanged (52.2-53.4 tok/s guided, 47.70 vs 47.67 at 8k), confirming the q=1 fallthrough is clean. Needle-8k recall 3/3, byte-identical. Production use: mf64/.

Bundle	Source checkpoint	Context	Functions	Bytes	Producer	Created
mf64-tp/lfm2_5_8b_a1b_decode_sym8_gather_mf64_tp	LiquidAI/LFM2.5-8B-A1B	16384	main+prefill	9,447,998,621	coreai-core 1.0.0b2	20260818T121930Z