FluidInference/lfm3-audio-2b-coreml

LFM3-Audio-2B for Core ML

LiquidAI/LFM3-Audio-2B converted to Core ML for Apple devices (macOS 15 / iOS 18 and newer): offline speech recognition (long-form included), LiveASR (streaming English transcription in 0.96 s chunks) and text-to-speech, all from one 2B audio language model. Run it with FluidAudio (LFM3AudioManager).

This is a modified version of the original model: the weights were converted to Core ML and the language model was quantized to int8 (block size 32). See Quantization and parity.

Core AI builds of the language model and the depth decoder (macOS 27 / iOS 27; faster ASR and TTS) are in FluidInference/lfm3-audio-2b-coreai; that backend uses the encoder, codec and tables from this repo.

Files

Part	File	Precision	Compute unit (default)
LM backbone, layers 0-11 / 12-23 (stateful; functions decode_q1, prefill_q256/64/16/13)	coreml/LFM3-A-C4160-int8b32.mlmodelc, coreml/LFM3-B-C4160-int8b32.mlmodelc	int8 weights (block 32), fp16 compute	GPU
LM head (128k vocab, per-chunk top-64)	coreml/LFM3-Head-int8b32.mlmodelc	int8	GPU
Causal audio encoder (functions n12 / n96 / n375, tokens per call)	coreml/AudioEncoder-fp16.mlmodelc	fp16	GPU (offline), ANE (LiveASR n12)
MaskGIT depth decoder (depth_in, depth_pass)	coreml/DepthDecoder-fp16.mlmodelc	fp16 (+ host fp32 sampling)	GPU
LiquidCodec decoder (f1 / f4 / f25, frames per call)	coreml/CodecDecoder-fp16.mlmodelc	fp16	ANE
Pack contract, tokenizer	assets/config.json, assets/tokenizer.json
Host tables: token embeddings, speech-frame embeddings, depth code embeddings, codec synthesis head	assets/embed_tokens.f16.bin, assets/codebook_embedding.f16.bin, assets/depth_tables.f16.bin, assets/codec_head.f32.bin	fp16 / fp32	CPU

ASR and LiveASR need about 2.5 GB of files; TTS adds the depth decoder, codec and tables (~0.4 GB). Context: 4096 positions. ane/ holds an experimental Neural Engine build of the LM that FluidAudio does not download; see Experimental: Neural Engine LM.

Use (Swift)

import FluidAudio

let manager = try await LFM3AudioManager.downloadAndLoad()  // .coreML; includeSpeech: true

// Offline ASR (16 kHz mono); language: "English", "French", ... or nil to infer
let result = try await manager.transcribe(samples)
print(result.text)

// LiveASR (English)
try await manager.startLiveTranscription()
let partial = try await manager.appendLiveAudio(microphoneChunk)
let final = try await manager.finishLiveTranscription()

// TTS (24 kHz), streamed as it is decoded
let speech = try await manager.synthesize(
    "Hello from LFM3 Audio.", voice: .female, language: "American English",
    onAudio: { chunk in /* play */ })

CLI: fluidaudiocli lfm3-audio transcribe|live|tts|benchmark.

Speed and accuracy

Apple M5 Pro (24 GB), macOS 27.0, FluidAudio release build, Core ML backend. Wall time includes model warm-up (the first calls after load), audio decoding excluded.

Task	Data	Result
Offline ASR	LibriSpeech test-clean, first 100 utterances (901 s)	WER 1.32%, RTFx 18.4; transcripts 100/100 identical to the original model (HF fp32 on MPS, greedy)
LiveASR (160 ms pieces)	LibriSpeech test-clean, first 20 utterances (157 s)	WER 1.46%, RTFx 10.4
TTS	20 phrases (72 s of speech, female voice)	RTFx 4.3-4.5 across runs, first audio after 37 ms (median); Parakeet TDT v3 round-trip WER 2.5%

Per speech frame (80 ms): LM 9.4 ms + depth decoder 4.9 ms on the GPU; the codec decodes on the ANE in parallel. Peak memory footprint for ASR: 1.9 GB.

Quantization and parity

• LM: int8 per-block-32 weights. Teacher-forced TTS hidden states match fp32 at cosine 0.9999; offline ASR transcripts are identical to the original model on the 100 utterances above. (An int4 build failed the speech-frame check.)
• Audio encoder, depth decoder and codec: fp16. Depth decoder codes are 99.6% identical to the original on 319 golden frames (sampling noise injected); codec waveform SNR 43-49 dB vs fp32.
• Host log-mel matches the original feature extractor to 8.3e-5.

Experimental: Neural Engine LM (ane/)

An all-ANE build of the language model (backbone + head), for testing. FluidAudio does not download or load it yet. On the M5 Pro it is slower than the GPU pack above. It is here for iPhone-class testing, where the ANE should match GPU latency at lower power; that has not been measured yet.

Part	File	Weights	ANE placement	ms per call (M5 Pro)
Layers 0-11	ane/LFM3-L0-12-q1-S1024-lut8g8.mlmodelc	LUT8, group 8	100%	5.77
Layers 12-23 + final norm	ane/LFM3-L12-24-q1-S1024-lut8g8.mlmodelc	LUT8, group 8	100%	6.01
Head (4 x 32,000 logits)	ane/LFM3-Head-lut8g8.mlmodelc	LUT8, group 8	100%	2.08

• Speed: one text token takes 14.43 ms (GPU pack ~10.8 ms), and one speech frame (backbone only) 12.03 ms (9.07 ms).
• Calls: decode only, one row per call, positions up to 1024. Input x is fp16 [1, 1, 2048], and the output h feeds the next part. The host supplies the RoPE tables (cos_g / sin_g, cos_s / sin_s) and the additive masks (mask_g, mask_s).
• State: the K/V caches and conv states are MLState. Full-attention layers write at the position given by the onehot input. ane/config.json lists each part's inputs and states.
• Head: returns raw logits in four 32,000-row slices. Argmax, top-k and token suppression run on the host.
• Parity vs fp32, measured with the same driver for both builds (GPU pack in parentheses):
  • ASR: 19/20 golden clips token-identical (20/20)
  • TTS teacher-forced hidden cosine: min 0.99969 (0.99983)
  • Depth codes: 97.15% identical (97.39%)
• ANE numerics:
  • Weight rows are pre-scaled by a power of two, and the scale is undone after each linear. This avoids fp16 underflow.
  • Sigmoid is computed as 0.5 + 0.5·tanh(x/2).
• Loading: the first load compiles for the ANE in about 5 s per part; cached loads take 35-60 ms.

License

LFM Open License v1.0 (Liquid AI). Commercial use by organizations with annual revenue of USD 10 million or more requires a separate license from Liquid AI; see the license for the full terms.

Changelog

Date	Files	Change	Issue	FluidAudio PR	HF commit
2026-10-29	all	Initial release: Core ML LM pack (int8 b32), audio encoder, depth decoder (per-pass functions), codec, host tables	FluidAudio #49	#1076	21447f69
2026-10-30	ane/*	Experimental all-ANE LM (LUT8 group 8, decode q=1, span 1024); not loaded by FluidAudio	FluidAudio #49	none	e5259314