mlboydaisuke/LFM3-1.7B-CoreAI

LFM3-1.7B — Apple Core AI (.aimodel)

LiquidAI's LFM3-1.7B converted to Apple's Core AI (iOS 27 / macOS 27): a 1.7B model that reasons before it answers and calls tools in XML, in two bundles with a 4,096-token context. The Mac bundle is one int8 .aimodel on the coreai-pipelined GPU engine that prefills a prompt 64 tokens at a time; the iPhone 18 Pro bundle is a Neural Engine build with fixed input shapes, compiled in advance for that phone's chip (h19p). CoreAIKit's ChatSession and Apple's LanguageModelSession drove both in the checks below; CoreAIKit's LFM3 support is on the kit's main branch since 9e9143bc, not yet in a release (Use it).

Source: LiquidAI/LFM3-1.7B at revision ac2586a34f1a1c03329dbdcdefabd8b6478ddaa7. Conversion code, gates and notes: coreai-model-zoo.

Bundles

path	size	for	what is in it
gpu-pipelined/lfm3_1_7b_decode_int8hu_block32_sym_pf64/	2.72 GB	Mac	lfm3_1_7b_decode_int8hu_block32_sym_pf64.aimodel, metadata.json, tokenizer/ (the source's tokenizer files; the chat template is the vendor's with eight rewrites that render the same string under transformers and parse in swift-jinja 2.3.2 and 2.3.6)
ios-h19p/	2.16 GB	iPhone 18 Pro	lfm3_1_7b_static_pal8_tanh_block_c4096.h19p.ane.aimodelc (compiled for the iPhone 18 Pro's Neural Engine, h19p; its resources.bin holds 1.48 GB of weights), metadata.json, tokenizer/ (the same four files)

Mac. int8hu: the MLP and conv-mixer linears int8 per-block-32, the head untied and int8 per-block-32 symmetric, the attention projections (q, k, v, o and the gate) and the q / k norms in fp32, everything else fp16. The .aimodel holds two functions over the same weights: main takes one token per step and prefill takes 64; the pipelined engine sends the prompt through prefill 64 tokens at a time while more than 64 tokens remain, and the last 1–64 tokens through main one at a time. On a Mac, Core AI compiles the .aimodel for the GPU the first time it loads and caches the result.

iPhone 18 Pro. The same 24 layers rebuilt the way Apple's iOS Neural Engine path requires (fixed shapes throughout), with each conv layer's input history kept in its own rows of the KV cache, the sliding window made from the engine's causal mask, 8-bit palettized weights (k-means, groups of 32; the attention gate fp16) and the tied head as an int8 table. It holds contexts of 256 to 4,096 tokens with queries of 8, 16 or 64 tokens, and the compile puts each of its 30 transformer functions in a Neural Engine region. It is compiled for h19p only: no other iPhone has a bundle here.

Speed

Mac (M4 Max, macOS 27.0), from a 1,024-token prompt: prefill 1,234 tok/s, 0.83 s to the first token, decode 169.5 tok/s (median of 8 runs, 2026-10-31; the published .aimodel, compiled on first load, run by the coreai-models version CoreAIKit pinned then: the path an app that uses CoreAIKit takes; llm-benchmark with 256 generated tokens, one process per run). Compiled ahead of time for the Mac GPU it is slower: decode 105.5 tok/s and 1.00 s to the first token in the same window.

In that window, alternated with it on the same 1,024 prompt tokens, MLX (mlx-vlm 0.7.11) decodes LiquidAI's 8-bit weights at 189.8 tok/s and llama.cpp (b12001) their Q8_0 at 180.2, both faster than this bundle, and their 4-bit forms faster still (286.9 and 242.1); their bf16 and F16 forms are slower (113.1 and 111.1). All six reach the first token in 0.23–0.26 s, and every form, this one included, gives transformers bf16's verdicts on the 8 questions below. What this port adds is a Neural Engine bundle for the iPhone and tool calls through Apple's LanguageModelSession.

iPhone 18 Pro (iOS 27.2), a 768-token prompt and 256 generated tokens: prefill 590.6 tok/s, decode 47.17 tok/s (medians of 6 trials), one turn 6.71 s (768 / prefill + 255 / decode, not timed), peak footprint 2.82 GB, thermal state nominal at the start and end of every run. In the gate app the first load on the phone took 34.9 s and later loads 4.4 s or less; a test app built on CoreAIKit (ChatSession(catalog:)), with the bundle at its own path, took 32.4 s on its first load, then 3.7 s and 0.4 s.

What was checked

Against transformers 5.20.2 on the same weights.

The Mac bundle, on a Mac (M4 Max, macOS 27.0):

• Teacher-forced on 17 sequences (8 questions with thinking off and on, a 1,648-token document) against transformers fp32, with the .aimodel compiled ahead of time for the Mac GPU and the prompts fed the way the engine feeds them (64-token chunks through prefill, the rest one token at a time): top-1 2639 / 2682 = 98.40 % (2,683 positions; the one where transformers' own top-2 gap is under 1e-3 is not counted); one token at a time throughout: 2637 / 2682. At every chunk boundary the logits agree with the one-token path to cos ≥ 0.99999.
• 8 questions, thinking off, greedy: 8 / 8, the same verdicts as transformers bf16.
• Token gate (16 greedy tokens vs the fp32 oracle): 16 / 16, also on a 215-token prompt that rides three chunks; a chat turn stops on <|im_end|>.
• Thinking on, the same 8 questions, greedy, 3,584-token budget, 3 runs of fresh processes: every run 8 / 8 — </think> closes after 40–237 tokens (median 56) and the answer after it is right (run on the main-only build of the same weights: prompts this short never reach prefill, and main gives the same values).
• A tool call, a tool returning a fresh random value, 3 runs each through CoreAIKit's KitLanguageModel and the zoo's ZooLanguageModel: 6 / 6 with the value in the answer verbatim.
• The bytes published here: the .aimodel is the export re-saved without its source locations (a new main.hash); run again, its teacher-forced values equal the export's at all 2,683 positions, fed in chunks and one token at a time, and the 8 questions, the token gate and the stop-token check give the same results.
• The kit's path (the .aimodel specialized on load, as timed above; the teacher-forced pass above was not repeated on this path): the 8 questions 8 / 8 with the same replies as the ahead-of-time build, the token gate 16 / 16 on both prompts, and the stop-token turn stops after 6 tokens (1+1=2; transformers fp32 stops after 2, at a step where its own top-2 gap is 0.005). Through CoreAIKit (ChatSession and KitLanguageModel), the 8 questions with thinking on (3 launches) and off, a tool round trip and a 3-turn chat pass.

The iPhone bundle, on an iPhone 18 Pro (iPhone19,2, iOS 27.2), through the fixed-shape (static-shape) engine of the coreai-models version CoreAIKit pinned then:

• Teacher-forced over the replies of the same 17 sequences (721 steps, up to 64 a sequence after its prompt) against transformers fp32: top-1 692 / 721 = 95.98 % (bar 93 %; every step's fp32 top-2 gap is at least 1e-3). The tokens equal those of Apple's own static engine on an iPhone 17 Pro on all 18 prompts, teacher-forced and generated, and a poisoned fixture fails, as it must.
• 8 questions, thinking off: 8 / 8, the verdicts of transformers bf16; the stop-token turn stops after 6 tokens.
• CoreAIKit's catalog path: ChatSession(catalog: "lfm3-1.7b") takes the ios variant and loads ios-h19p/ (copied into the kit's model store at the path a download would use, so no download ran) on the static-shape engine. Thinking on, the 8 questions in 3 fresh launches: 8 / 8 each, </think> inside the budget and the reasoning out of the answer. Thinking off: 8 / 8. A tool round trip through KitLanguageModel and LanguageModelSession: 3 / 3 with the value verbatim. With the default configuration, "What is 17 + 25? Answer briefly." gets "17 + 25 = 42.", and a 3-turn chat recalls its first turn, every turn ending on the stop token.
• With thinking off, 1 of the 8 questions ("How do you say 'thank you' in French?") still opens <think> and reasons for 68 tokens before its answer, which gives "Merci"; transformers bf16 and the Mac bundle answer it directly. Apple's engine on the iPhone 17 Pro does the same, so this comes from the Neural Engine export itself, not from CoreAIKit's engine.

The full table, the transcripts and how the fp16 activations set a floor on this model are on the zoo card and in the port notes.

Use it

What runs today without CoreAIKit is at the end of this section. CoreAIKit's support for LFM3 — the XML tool calls, reading a reply that starts inside the thinking block, the thinking switch (enableThinking) and the lfm3-1.7b catalog entry — is on the kit's main branch since 9e9143bc (coreai-kit#83), not yet in a CoreAIKit release (0.7.3 does not support LFM3). The CoreAIKit checks above ran a kit build that has it; with the kit's main branch, on a Mac or an iPhone 18 Pro:

import CoreAIKit

let chat = try await ChatSession(catalog: "lfm3-1.7b")
for try await event in await chat.streamResponse(to: "What is 17 + 25? Answer briefly.") {
    switch event {
    case .thinking(let delta): print(delta, terminator: "")   // the reasoning comes first
    case .response(let delta): print(delta, terminator: "")   // then the answer
    default: break
    }
}

Thinking is on by default (the chat template ends the prompt with <think>); maxResponseTokens covers the reasoning and the answer together. ChatSession.Configuration.enableThinking = false renders the template's thinking-off prompt; LiquidAI notes that turning thinking off "can be inconsistent, especially for reasoning-heavy tasks".

Tool calls come out as XML (<tool_call> / <function=NAME> / <parameter=KEY>). CoreAIKit's KitLanguageModel renders the tools the way the chat template does and parses these calls for Apple's LanguageModelSession — the tool-call checks above.

On an iPhone the kit takes ios-<arch>/ for the architecture Core AI reports for the phone (AIModel.deviceArchitectureName: h19p on the iPhone 18 Pro) and falls back to ios/. This repo has ios-h19p/ only, so on any other iPhone (the iPhone 17 Pro, which reports h18p, included) the kit finds no bundle and throws variantNotFound before it downloads anything. Both apps that ran the iPhone checks carried the com.apple.developer.kernel.increased-memory-limit entitlement (peak footprint 2.82 GB in the gate app, up to 2.94 GB through ChatSession); a run without it was not tried.

Without the kit: the Mac gates ran llm-runner built from the zoo's fork of apple/coreai-models, whose pipelined engine carries the conv state as an extra state and finds the prefill function by name, with COREAI_CHUNK_THRESHOLD=1 and --inference-engine-variant coreai-pipelined --warmup off; the iPhone checks ran the StaticShapeEngine of that fork's tag 0.2.8-zoo, the one CoreAIKit pinned then. The Mac tool-call checks also ran through the zoo's ZooLanguageModel (swift/Sources/ZooFMProvider in the zoo; it depends on the coreai-models package, not on CoreAIKit).

Limits

• The engine cannot rewind the Mac bundle's conv state by position. Since coreai-models 0.2.9-zoo (CoreAIKit's pin since coreai-kit#82), it checkpoints that state at the end of the prompt's last message, on the 64-token chunk grid, and CoreAIKit's ChatSession resumes each later turn from the checkpoint instead of prefilling the whole history; a history shorter than one chunk is prefilled again (in a three-turn chat on the kit's main branch, turn 2, after a 32-token first prompt, prefilled all of its 71 tokens, and turn 3 reused 64 of its 103). The iPhone bundle keeps the conv history in the KV cache, and in the 3-turn chat its engine reused the earlier turns' prompt tokens (32, then 70).
• On the Mac the engine always runs the last prompt token through the one-token main function, so a prompt's final 1–64 tokens run one at a time at about the decode speed: a 1,024-token prompt runs 15 chunks and then 64 single steps.
• 4,096-token context in both bundles.
• The iPhone bundle is for the iPhone 18 Pro only; the iPhone 17 Pro has no bundle here. The Mac bundle is not for iPhones: on an iPhone 18 Pro the main-only .aimodel, specializing on the phone (device JIT), had not built its engine 300 s after launch (the cause is not isolated), and on an iPhone 17 Pro the same int8 weights compiled ahead of time for its GPU decoded at 9.25 tok/s (median of 4 trials at thermal state serious; 10.5 in the first run, at fair), that compile holding fp16 copies of the int8 linears.
• On a Mac, Core AI named an ahead-of-time compile's cache entry after its graph and not its weights, so a second compile of the same graph with other weights ran with the first one's weights (not checked on an iPhone): an app that switches between such compiles should delete that compile-cache entry first. The Mac bundle is a JIT .aimodel, whose cache entry name covers its weights.

License

The weights are under the LFM Open License v1.0 (LICENSE, carried from LiquidAI/LFM3-1.7B): Apache-2.0-style grants, with §5: commercial use by a legal entity at or above US$10 M annual revenue is not licensed (the threshold does not apply to a qualified non-profit's non-commercial or research use). Converted from LiquidAI's weights; not affiliated with Apple or LiquidAI.