visible-cx/LFM2.5-2.6B-CoreAI

LFM2.5-2.6B — Core AI (.aimodel)

LiquidAI/LFM2.5-2.6B converted to Core AI .aimodel bundles for Apple silicon by visible-cx. These are derivative artifacts: Liquid AI's weights re-expressed as a Core AI graph with int8 block-32 symmetric weight quantization and a two-entrypoint (decode + chunked-prefill) function map. They load through Core AI on macOS and are not usable by PyTorch, GGUF or MLX.

Of the dense LFM2.5 bundles published here, this is the strongest on guided structured-output work and the only one qualified for long context: 3/3 verbatim needle recall at 14,566 tokens, sustaining 32 tok/s for under half a gigabyte of in-process footprint.

Provenance

Base checkpoint	LiquidAI/LFM2.5-2.6B
Recipe	export_lfm2_multifunction.py int8hu --head-sym --chunk 64
Toolchain base	apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8 + the coreai-model-zoo python overlay
Toolchain	coreai-torch 0.4.1, coreai-core 1.0.0b2, coreai-opt 0.2.1, torch 2.9.0
Producer fingerprint	coreai-core 1.0.0b2 on every inner <name>.aimodel/metadata.json
Weight format	int8, per-K-block-32, symmetric; symmetric head (--head-sym)
Vocab	128,000
Export functions	main (S=1 decode) + prefill (S=64 chunked prefill), function_map: {"main": ["main", "prefill"]}, weights deduplicated across entrypoints

mf64 in the bundle name means multifunction with a 64-wide prefill; the prefill function costs well under a megabyte.

Chat template. LFM2.5-2.6B is an always-thinking model. The template shipped in every bundle here terminates the reasoning block in the generation prompt:

{{- "<|im_start|>assistant\n<think></think>\n" -}}

An unterminated block (…assistant\n<think>) causes the model to spend the entire generation budget inside <think>, which a host routes to a reasoning channel and never to the response — 684–919 tokens per item, with no visible output. If you rebuild from the recipe, apply the same termination.

Contents

Path	Bytes	Manifest context	Functions
gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64	3,655,243,493	4096	main + prefill
ctx8192/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64	3,655,243,501	8192	main + prefill
ctx16384/gpu-pipelined/lfm2_5_2_6b_decode_int8hu_block32_sym_mf64	3,655,243,498	16384	main + prefill

Each folder holds <name>.aimodel/ (main.mlirb ≈ 3.64 GB, main.hash, asset metadata.json), a bundle-level metadata.json, and tokenizer/ (tokenizer.json, tokenizer_config.json, generation_config.json, chat_template.jinja).

The three folders hold the same weights and the same graph — function signatures, state descriptors and peak export RSS are identical at 4096, 8192 and 16384. --max-ctx changes exactly one thing: language.max_context_length in the bundle manifest. The small byte differences between folders are conversion nondeterminism, not content. Pick the folder whose manifest integer matches the window you intend to run.

Stop token: eos_token = "<|im_end|>" in all three folders. Clean self-stop on every measured sample.

Requirements

• Apple silicon Mac, Core AI runtime.
• Engine contract: 2 inputs. input_ids, position_ids → logits. No static inputs, no per-step mask. Runs on both the pipelined engine and the sequential (logits-capable) engine, which is what makes grammar-constrained decoding available.
• States: keyCache / valueCache Float16, 8 × 1 × 8 × ? × 64 plus convState Float16, 22 × 1 × 2048 × 2. The sequence dim is dynamic, so the runtime resolves a GrowingKVCache (initial capacity 256, doubling) rather than allocating the manifest maximum up front. convState is fixed-size and does not scale with context.
• KV cost: 16,384 bytes per token of context (fp16) — 67 MB at 4096, 134 MB at 8192, 268 MB at 16384.
• Memory envelope, 16 GB Mac (Metal recommendedMaxWorkingSetSize ≈ 10.7 GB): weights 3.64 GB resident, so KV is not the binding constraint at any context this bundle declares. Measured max RSS 6.71 GB on guided decoding and 7.30 GB at 15k context; peak in-process footprint 0.48 GB and 0.46 GB respectively — essentially flat in context. Minimum practical machine memory: 16 GB, at any declared context including 16384.
• The bundle manifest declares runtime_env COREAI_CHUNK_THRESHOLD=1.

Performance

Measured on a 16 GB Apple silicon Mac (M2 Pro, macOS 27 beta): 10-sample harness, guided JSON-constrained decoding against a fixed schema, greedy, sequential engine, reset() between samples, 128-token cap. Load excluded from s/row; sample 1 excluded as a cache-warm outlier.

Cold load	18.4 s
Guided JSON parse	10/10
Enum-clean	10/10
s/row (long samples)	3.66
s/row (short samples)	3.06
Decode	38.1–40.0 tok/s
TTFT	0.47–2.29 s
Peak footprint	0.48 GB
Max RSS	6.71 GB

For scale on the same machine and harness: LFM2.5-350M runs 0.84/0.55 s/row and LFM2.5-1.2B 1.89/1.37 s/row. The 2.6B is ≈2× the 1.2B, which is what its parameter count predicts, and it produces the most specific free-text fields of the three.

Long context

Needle-in-haystack: 3/3 verbatim at both 8k and 15k. Three distinctive facts planted at 10% / 50% / 90% of the filler, strict scoring (a fact counts only if the distinctive entity comes back correct). All three returned at 7,813 tokens and all three at 14,566 tokens, verbatim, including the date.

probe	prompt tokens	TTFT	decode	wall	peak footprint
needle 8k	7,813	13.78 s	35.8 tok/s	15.4 s	0.34 GB
needle 15k	14,566	27.42 s	32.2 tok/s	29.1 s	0.46 GB

Free-form generation from a fixed prompt at three depths, 900-token cap (the model self-stopped inside it every time):

depth	prompt tokens	TTFT	decode	generated	wall	peak footprint
3.4k	3,249	6.88 s	35.3 tok/s	847	29.5 s	0.28 GB
8k	7,673	14.84 s	36.6 tok/s	715	36.0 s	0.47 GB
12k	11,643	22.24 s	32.9 tok/s	743	44.4 s	0.46 GB

Decode barely moves with depth. 40.0 tok/s at 2.3k → 35.3 at 3.2k → 36.6 at 7.7k → 32.9 at 11.6k: an 18% decay across a 5× context increase. It sustains 32–33 tok/s at 11.6–14.5k tokens with under half a gigabyte of in-process footprint. High context costs prefill time and almost nothing else — TTFT scales cleanly (4.8 → 6.9 → 14.8 → 22.2 → 27.4 s) while footprint stays flat. The KV term is GrowingKVCache-dynamic and small at this size (16 KB/token → 268 MB at 16384).

Status

Artifact	Status
gpu-pipelined/…_mf64 (ctx 4096)	SHIP — measured: 10/10 guided parse and enum-clean, 3.66/3.06 s/row, 38.1–40.0 tok/s.
ctx8192/…_mf64	QUALIFIED AT DEPTH — same weights and graph; measured: 3/3 verbatim needle recall at 7,813 tokens, 36.6 tok/s at 8k, 0.47 GB peak footprint.
ctx16384/…_mf64	QUALIFIED AT DEPTH — 3/3 verbatim needle recall at 14,566 tokens at 32.2 tok/s, 0.46 GB peak footprint, 7.30 GB max RSS. Recommended for long-context work.

No oracle or PSNR gate has been run against a PyTorch reference. Qualification is behavioural (parse rate, enum conformance, content sanity, clean stop, needle recall), not a numerics gate.

License

LiquidAI/LFM2.5-2.6B is released under the LFM Open License v1.0 (lfm1.0). These bundles are a derivative of that checkpoint and the same licence and its obligations travel with them — see the upstream licence. Anyone redistributing these files should redistribute the licence with them and comply with its terms. Nothing here relicenses Liquid AI's weights; the contribution is the conversion recipe and the qualification evidence.