mashbox-labs/LFM3-1.7B-CoreML

LFM3-1.7B for the Apple Neural Engine

This is LFM3-1.7B by Liquid AI as a Core ML model that runs on the Apple Neural Engine.

The weights come from Liquid AI's own quantization-aware release, LFM3-1.7B-QAD-Q4_0.gguf in LiquidAI/LFM3-1.7B-GGUF: 4-bit weights for the layers and 6-bit weights for the shared embedding and output layer. Every 4-bit and 6-bit value and every scale is carried over exactly, so the model computes with exactly Liquid's weights. MASHBOX Labs rebuilt the network as one stateful Core ML program for MLComputeUnits.cpuAndNeuralEngine. It computes in float16.

This is not an official Liquid AI release, and it is not affiliated with or endorsed by Apple.

Measured

On a Mac with an Apple M5 (macOS 27.2):

Size	1,059 MB (either .mlpackage), 4.9 bits per weight. Liquid's GGUF file is 1,049.6 MB
Context	8,192 tokens with the -8k files. A pass key hidden in a 7,270-token prompt is recalled on the Neural Engine, and the compute-op placement is the same as the 2,048-token build (12,304 of 13,384 ops). The prompt runs at 44 tokens/s on a busy Mac; the 8,192-token build was not timed on an idle one
Neural Engine	91.9 % of the compute ops in decode run on the Neural Engine (12,304 of 13,384, from MLComputePlan), for both context sizes. The other 1,080 run on the CPU and look up the input token's embedding row
Same output as Liquid's weights	98.22 % next-token agreement on the Neural Engine (2,761 of 2,811 positions of 44 prompts) with Liquid's weights run in full precision (PyTorch, float32). The same weights run in float16 PyTorch agree 99.89 %. The same Core ML model on the CPU agrees on all 400 positions of the first 6 prompts, where the Neural Engine agrees on 393, so the gap comes from the Neural Engine's arithmetic, not from the weights
Greedy text	25 of 44 prompts give word-for-word the same greedy reply (up to 160 tokens) as the full-precision reference
Speed	2,048-token build: 11.1 tokens/s decoding (90.0 ms per decode call). The prompt runs at 47.7 tokens/s (167.9 ms per 8-token prefill call). A verify call takes 191.5 ms. One conversation, measured once on an idle Mac (load average 0.7) as the median of 64 decode, 16 prefill and 32 verify calls
Memory	235 MB process footprint after loading and generating (peak 337 MB, phys_footprint). This counts the app's own process only, not memory the system holds for the Neural Engine
First load	158 s the first time on a device, while macOS compiles the model for the Neural Engine. Later loads take about 2 s

Files

File	What it is
LFM3-1.7B-MASHBOX-4.9bpw-8k.mlmodelc/	the compiled model with an 8,192-token context. It loads directly, with no compile step in your app. The example uses this one
LFM3-1.7B-MASHBOX-4.9bpw-8k.mlpackage	the same model as a source package, for Xcode
LFM3-1.7B-MASHBOX-4.9bpw.mlmodelc/	the same weights with a 2,048-token context: a smaller cache for short prompts
LFM3-1.7B-MASHBOX-4.9bpw.mlpackage	its source package
tokenizer.json, tokenizer_config.json, special_tokens_map.json, chat_template.jinja	from LiquidAI/LFM3-1.7B, unchanged
LICENSE	Liquid AI's licence, from LiquidAI/LFM3-1.7B, unchanged
example/	a small Swift command-line program that loads the model and generates text

Run it

You need a Mac with Apple silicon, macOS 15 or later, and Xcode (or the Xcode command-line tools) with Swift 6.

python3 -m venv hf-env && source hf-env/bin/activate
pip install -U huggingface_hub
hf download mashbox-labs/LFM3-1.7B-CoreML --local-dir LFM3-1.7B-CoreML
cd LFM3-1.7B-CoreML/example
swift build -c release
.build/release/lfm3-generate .. "Write two sentences about the ocean."

The program loads the model and formats your message with the model's chat template (thinking off). It then decodes greedily and prints the reply as it is generated. Afterwards it prints the prompt time and the tokens per second. On the M5 it prints about 10 to 11 tokens/s, because each step also builds the inputs and turns tokens into text.

• --system "..." adds a system prompt.
• --max-new N caps the reply (default 256 tokens).

The download is 4.2 GB because the repo holds two context sizes, each compiled and as a package. The example needs only the compiled 8,192-token one: add --exclude "*.mlpackage/*" --exclude "LFM3-1.7B-MASHBOX-4.9bpw.mlmodelc/*" to hf download to fetch 1.1 GB.

The first load on a device is slow because macOS compiles the model for the Neural Engine and caches the result. Later loads take about 2 s.

Using it in your own app

The model has three functions that share one state:

• prefill: 8 tokens per call.
• decode: 1 token per call.
• verify: 8 tokens per call, with logits for every position.

Make the state once per conversation from decode (decodeModel.makeState()) and pass it to every call. example/Sources/lfm3-generate/main.swift shows how every input is built.

Input	Shape	Meaning
input_ids	(1, S) int32	token ids: S = 8 for prefill, 1 for decode, 8 for verify
valid	(1, 1, 1, S) float16	1 for a real token, 0 for padding
E	(1, S, ctx) float16	one-hot cache position of each real token (all zero for padding). ctx is 8192 for the -8k files and 2048 for the others; the example reads it from this input's shape
mask_f, mask_s	(1, 1, S, ctx) float16	additive attention masks (0 or -10000), one for the full-attention layers and one for the sliding-window layers (window 256)
cos_f, sin_f, cos_s, sin_s	(1, 1, 128, S) float16	rotary tables for the full and the sliding layers
sel	(32, 6) float16	picks the 6 newest short-convolution inputs. Column i is one-hot at row 26 + i, or at row 26 - (8 - k) + i after a verify call that kept only its first k tokens

The output is logits. For prefill and decode it has shape (1, 128000) and holds the last position. For verify it has shape (1, 8, 128000).

• The context is 8,192 tokens with the -8k files and 2,048 tokens with the others. The weights are the same; only the cache size differs.
• Use the chat template from the original model with enable_thinking=false, so the assistant turn starts with an empty <think></think>. The reply ends at <|im_end|> (id 124900).
• verify is there for speculative decoding: it checks up to 7 drafted tokens in one call. A speculative greedy run can differ from a plain greedy run where the two best tokens are almost tied. This happens because verify and decode are compiled as separate programs that compute in float16.

Tested

	Tested	Not tested
Device	Mac with Apple M5, macOS 27.2	iPhone, iPad, other Macs, older macOS
Prompts	English	the other languages LFM3-1.7B supports (Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai)

Credits and licence

• Model: LFM3-1.7B by Liquid AI (LiquidAI/LFM3-1.7B).
• Weights: Liquid AI's quantization-aware release (LiquidAI/LFM3-1.7B-GGUF, LFM3-1.7B-QAD-Q4_0.gguf).
• Licence: LFM Open License v1.0 (lfm1.0), as Liquid AI states it for LFM3-1.7B. The upstream LICENSE file is included unchanged.
• Changes by MASHBOX Labs: the model is converted to Core ML and rebuilt as a stateful Neural Engine program. Liquid's quantized weights are carried over exactly. Its small float32 tensors (normalization and convolution-filter weights) are stored in float16.