mashbox-labs/Phonon-2.5-English-CoreML

Phonon-2.5 English for the Apple Neural Engine

This is Phonon-2.5 English by Fermion Research as Core ML models for Apple devices. The speech encoder runs on the Apple Neural Engine. The files load in FluidAudio the same way Phonon-2 does.

Phonon-2.5 English is derived from NVIDIA's Parakeet TDT 0.6B v3. The encoder's compressed weights are carried over exactly from Fermion Research's release, with no further compression. The decoder and joint network are built from the release's own tables. The audio front end is exported from the model's own feature-extractor configuration.

This is not an official Fermion Research release, and it is not affiliated with or endorsed by Apple, NVIDIA or FluidInference.

Measured

On a Mac with an Apple M5 (macOS 27.2), with FluidAudio 0.18.0 loading the files through AsrModels.loadLocal and the encoder on .cpuAndNeuralEngine. The reference is Fermion Research's own reference_transformers.py (PyTorch, float32, whole clip at once). Word error rates use the Whisper English text normalizer.

Size	379.6 MB in total. The encoder is 341.1 MB, the decoder 23.6 MB, the joint network 12.7 MB and the audio front end 2.0 MB
Neural Engine	99.5 % of the encoder's compute ops run on the Neural Engine (1,378 of 1,385, from MLComputePlan), with 100 % of its estimated cost. The CPU's busy time is 1.5 % of the wall-clock time while the encoder runs
Encoder output	Cosine similarity 0.99973 to 0.99987 with the float32 reference on the same input (normalized RMS error 0.017 to 0.023, 15 s window, 4 clips)

Word error rate (%) against the reference, on clips from the test sets:

Set	Clips	Reference	This model	Difference	Same text as the reference
LibriSpeech test-clean	200	1.917	1.940	+0.023	196 of 200
LibriSpeech test-other	200	4.853	4.769	-0.085	193 of 200
FLEURS English test	100	5.733	5.733	0.000	93 of 100
Long clips, 52 to 118 s (joined LibriSpeech test utterances)	12	2.385	3.143	+0.757	1 of 12

No clip gave an empty transcript or an error.

Long audio is less accurate than the reference. Our goal was within +0.10 points of the reference on every set, and the long clips miss it (+0.757). On 2 of the 12 long clips, the last words also differ from the reference's. Nothing is cut off: every long transcript has 99 to 101 % of the reference's word count. The likely cause is that FluidAudio transcribes long audio in 15 s windows, while the reference hears the whole clip at once. The encoder passes the tensor check above on a single window. Audio up to 15 s runs as one window.

FluidAudio's AsrModels.load (which also uses the encoder's 5 s and 10 s functions) scored +0.135 (test-clean), -0.085 (test-other), +0.043 (FLEURS) and +0.644 (long clips) on the same clips.

Speed, against FluidInference/phonon-2-coreml on the same Mac with the same program, alternating, three rounds:

	This model	Phonon-2
All 512 clips above (4,875.5 s of audio), median of 3	29.99 s (163x real time)	29.77 s
One 10 s clip, median of 15	64.9 ms	63.7 ms
One 30 s clip, median of 15	135.5 ms	137.0 ms
First load on a device	52.4 s	52.6 s
Later loads	0.17 s	0.17 s

The first load is slow because macOS compiles the models for the Neural Engine and caches the result.

Files

File	What it is
Preprocessor.mlmodelc/	audio front end: 16 kHz audio (up to 15 s) to a 128-band log-mel spectrogram. Runs on the CPU in float32
Encoder.mlmodelc/	speech encoder for the Neural Engine. Functions enc_15000ms (the default), enc_10160ms and enc_5040ms
Decoder.mlmodelc/, JointDecisionv3.mlmodelc/	TDT decoder and joint network
parakeet_vocab.json	the model's 8,192 tokens
LICENSE-WEIGHTS-CC-BY-4.0.txt, NOTICE	from FermionResearch/Phonon-2.5-English, unchanged
example/	a small Swift command-line program that transcribes audio files

Run it

You need a Mac with Apple silicon, macOS 15 or later, and Swift 6 (Xcode or the Xcode command-line tools).

python3 -m venv hf-env && source hf-env/bin/activate
pip install -U huggingface_hub
hf download mashbox-labs/Phonon-2.5-English-CoreML --local-dir phonon-2
cd phonon-2/example
swift build -c release
.build/release/phonon-transcribe .. recording.wav

It prints the load time, then each file's transcript and how long it took.

Using it in your own app

Add FluidAudio 0.18.0 and load the folder as the Phonon-2 model version:

import FluidAudio

let models = try AsrModels.loadLocal(from: modelFolder, version: .phonon2, encoderComputeUnits: .cpuAndNeuralEngine)
let asr = AsrManager(config: .default)
try await asr.loadModels(models)
var state = TdtDecoderState.make(decoderLayers: await asr.decoderLayerCount)
let result = try await asr.transcribe(audioURL, decoderState: &state)
print(result.text)

• Use .phonon2, not .phonon25: FluidAudio runs .phonon25 on the GPU only.
• AsrModels.load(from:version:) reads its files from a folder named phonon-2 beside the folder you pass, and downloads Phonon-2 there if it is missing. To use it with these files, keep them in a folder named phonon-2 and pass that folder. loadLocal reads only the folder you pass.

Tested

	Tested	Not tested
Device	Mac with Apple M5, macOS 27.2	iPhone, iPad, other Macs, older macOS
Audio	English read speech (LibriSpeech) and FLEURS English	conversational, noisy or far-field audio; other languages

Credits and licence

• Model: Phonon-2.5 English by Fermion Research (FermionResearch/Phonon-2.5-English), derived from NVIDIA's Parakeet TDT 0.6B v3 (nvidia/parakeet-tdt-0.6b-v3).
• Licence: CC-BY-4.0, as Fermion Research states it for the weights. Fermion Research's LICENSE-WEIGHTS-CC-BY-4.0.txt and NOTICE are included unchanged; NOTICE lists the base model, its changes and the training data.
• Runtime: FluidAudio by FluidInference (Apache-2.0), which these files are built to load in. parakeet_vocab.json is in FluidAudio's vocabulary format.
• Changes by MASHBOX Labs: the model is converted to Core ML, with the encoder built for the Neural Engine. The encoder's compressed weights are carried over exactly. The decoder and joint network are stored in float16.