Phonon-2 for LiteRT

This repository is a conversion of FermionResearch/Phonon-2 (English speech-to-text) to LiteRT, made with litert-torch 0.9.4 and ai-edge-quantizer 0.10.0 for LiteRT 2.3.0: on a Galaxy S26 its float16 encoder, compiled ahead of time for the phone's NPU (Qualcomm SM8850), transcribes 178× faster than real time, and on a Pixel 11 Pro the same float16 graphs, in one file on the GPU, 26× (655 LibriSpeech test-clean utterances, one after another, measured 2026-11-08). On all of LibriSpeech test-clean the word error rate on the Galaxy S26's NPU is 1.726 %, against 1.736 % for our float32 PyTorch run of the same windows. Speed still being tuned (updates follow).

FermionResearch/Phonon-2 keeps the architecture of NVIDIA's parakeet-tdt-0.6b-v3 and packs its weights into a 164 MB download: the linear weights of the encoder are re-trained to five values per output row ({0, ±lo, ±hi}), and the other large weights are 6-bit tables. These LiteRT files do not keep that packing. The large weights are expanded to float16 (GPU, NPU) or int8 (CPU), so the files are larger: the files each path below runs (encoder, mel and decoder files) come to 1.44 GB on the Galaxy S26's NPU, 1.48 GB on the GPU, 0.93 GB on the CPU, 3.68 GB for the first release's NPU files compiled on the phone and 1.95 GB for the desktop script. What they add is the phone's NPU and GPU through LiteRT. For the 164 MB form, Fermion Research publishes its own Core ML and Android CPU builds (links on its card).

The graphs are built by the same code as litert-community/parakeet-ultra's, and the 10, 20 and 30 s mel files are the same bytes.

Files

The first release's model files, runner and scripts are unchanged. Added after it: the three one-file encoders, the 5, 7.5, 15 and 25 s mel files, and the runner ultra_asr_runner_p8 with its build script.

file	bytes	what	backend
phonon2_encoder_all_f16_sm8850.tflite	1,403,913,392	the f16 encoder for all seven window lengths (5, 7.5, 10, 15, 20, 25, 30 s) in one file, compiled ahead of time for the Qualcomm SM8850 (QAIRT 2.50.0.260828): the seven graphs are one compiled program, their weights stored once	NPU (SM8850)
phonon2_encoder_all_f16.tflite	1,450,789,936	the same seven f16 graphs in one file, not compiled: a buffer that is the same bytes in several graphs is stored once (1.45 GB of buffers in place of the seven graphs' 8.46 GB)	GPU
phonon2_encoder_all_i8.tflite	902,202,176	the i8 encoder for all seven window lengths in one file: a buffer that is the same bytes in several graphs is stored once (0.86 GB of buffers in place of the seven graphs' 4.38 GB)	CPU
phonon2_encoder_10s_i8.tflite	616,306,528	encoder for windows up to 10 s; int8 weights, one scale per output channel, in the linear layers and the two 1×1 convolutions of the subsampling, with int8 activations computed at run time; the depthwise convolutions and the 24 relative-position tables stay float32	CPU
phonon2_encoder_20s_i8.tflite	640,882,528	the same, up to 20 s	CPU
phonon2_encoder_30s_i8.tflite	665,458,528	the same, up to 30 s	CPU
phonon2_encoder_10s_f16.tflite	1,194,680,752	encoder for windows up to 10 s; float16 weights in the linear and convolution layers, LayerNorm inputs rescaled so that float16 arithmetic does not overflow	GPU, NPU
phonon2_encoder_20s_f16.tflite	1,219,256,752	the same, up to 20 s	GPU, NPU
phonon2_encoder_30s_f16.tflite	1,243,832,752	the same, up to 30 s	GPU, NPU
phonon2_mel_10s.tflite	1,608,068	log-mel front end, 10 s of 16 kHz audio (no weights: the same bytes as litert-community/parakeet-ultra's and litert-community/parakeet-redux's)	CPU
phonon2_mel_20s.tflite	2,252,068	the same, 20 s	CPU
phonon2_mel_30s.tflite	2,896,076	the same, 30 s	CPU
phonon2_mel_5s.tflite	1,288,644	log-mel front end for 80,640 samples (5 s) of 16 kHz audio	CPU
phonon2_mel_7.5s.tflite	1,448,356	the same for 120,320 samples (7.5 s)	CPU
phonon2_mel_15s.tflite	1,932,644	the same for 240,640 samples (15 s)	CPU
phonon2_mel_25s.tflite	2,576,644	the same for 400,640 samples (25 s)	CPU
phonon2_decoder_i8.tflite	17,866,768	TDT prediction network (predict) and joint network (joint), int8 weights	CPU
tokenizer.json	1,159,960	the tokenizer of nvidia/parakeet-tdt-0.6b-v3 (Phonon-2 keeps it)	host
ultra_asr_runner_p8.cc, build_android_p8.sh		the command-line runner for Android (LiteRT C API) that runs the one-file encoders and the seven window lengths, and the script that builds it and ultra_asr_runner_p
ultra_asr_runner_p.cc, build_android.sh		a command-line runner for Android (LiteRT C API) and its build script: parakeet-ultra's files, unchanged (the runner reads the model files from its arguments)
tdt_text.py		turns the runner's token ids into text (parakeet-ultra's file, unchanged)	host

Each encoder graph holds the whole encoder; the window lengths only save the work of padding short audio.

How it works

audio (16 kHz mono) → 30 s windows → mel front end → encoder → greedy TDT decoding → text

• Windows. The audio is cut into consecutive windows of 479,840 samples (30 s); the last one is shorter. A window under 320 samples (two mel frames) is dropped and has no text. Each window is transcribed on its own, and the window texts are joined with one space. Long audio is cut at fixed 30 s points, not at pauses, so a word that crosses a cut can be split, lost or repeated.
• Buckets. Each window goes to the smallest graph pair that holds it: up to 80,640 samples → the 5 s mel and encoder, up to 120,320 → 7.5 s, 160,000 → 10 s, 240,640 → 15 s, 320,000 → 20 s, 400,640 → 25 s, else 30 s (the first release's three-file paths: 10, 20 and 30 s only). The shorter graphs only skip the work on padding: in float32 the seven window lengths gave our PyTorch reference's tokens on all 83 windows of our 82-utterance test subset. On the 655-utterance LibriSpeech runs of the performance table the windows went 279 / 141 / 84 / 95 / 36 / 15 / 8 to the 5 / 7.5 / 10 / 15 / 20 / 25 / 30 s graphs.
• One file, seven graphs. In the three all files each window length is a signature of its own (enc5, enc7.5, enc10, enc15, enc20, enc25, enc30), with the inputs and output of the encoder below. The sm8850 file holds the seven compiled for the NPU of that chip, so the phone compiles nothing. The GPU file is the uncompiled form; the runner's --gpu-share-constants 1 keeps one copy of a constant that several graphs read. The i8 file holds the seven i8 graphs for the CPU.
• Mel front end (phonon2_mel_*, inputs audio [1, samples] float32 and n_valid [1] int32): pre-emphasis, STFT (512 points, 400-sample Hann window, hop 160), 128 Slaney mel bins, log and per-window normalization over the valid frames, as transformers' ParakeetFeatureExtractor. Outputs features [1, 128, T], mel_mask [1, T] and frame_mask [1, T / 8] (T = 504 / 752 / 1,000 / 1,504 / 2,000 / 2,504 / 3,000 for 5 / 7.5 / 10 / 15 / 20 / 25 / 30 s).
• Encoder (phonon2_encoder_*, inputs features and mel_mask as the mel graph gives them): output enc [1, T / 8, 640]; rows past the valid frames are undefined.
• Decoding (on the host, phonon2_decoder_i8): predict (token [1, 1] int32, h and c [2, 1, 640]) → dec_out [1, 640] and the new h, c; joint (one enc row [1, 640], dec_out) → 8,198 logits = 8,193 token logits (8192 = blank) + 5 duration logits (0–4 frames). Start from blank with zero state; at each step take the arg-max token and duration; a blank with duration 0 moves one frame; a non-blank token is emitted and fed to predict; ten tokens on one frame move one frame; token 3 ends the window; stop at the last valid frame (the sum of frame_mask) or after 3,750 steps.
• Pipelining (the Android runners' --pipeline). With --pipeline 2 (ultra_asr_runner_p8) the mel front end runs on one thread, the encoder on a second and the decoding on the main thread, window after window in the input order, with up to two windows (--pipeline-depth) waiting between two stages, also across files. With --pipeline 1 (both runners) a second thread runs the mel front end and the encoder of the next windows (up to two ahead) while the main thread decodes; --pipeline 0 runs everything on one thread, window after window. Each window's decoding still starts from blank with zero state, so the setting does not change the tokens: on the Galaxy S26's NPU, --pipeline 2 gave the same tokens, frames and durations as --pipeline 1 (with an earlier build of the runner and a four-graph f16 file compiled ahead of time, neither in this repository), and --pipeline 1 the same as --pipeline 0 (with the three f16 files compiled on the phone), on all 658 windows of our 655-utterance runs. With a single window there is nothing to overlap: the gain comes from runs of several windows or files.
• Text. Decode each window's token ids with tokenizer.json, special tokens skipped and repeated ids kept (Tokenizer.decode(ids, skip_special_tokens=True)). Id 3 (<|endoftext|>) is not flagged special in tokenizer.json, so a window that ended on it would print the literal token; no window did in our runs.

Use it

1. Download

Each path below runs one encoder form with the mel files, the decoder and the tokenizer; the runners are built from the repository's sources. This block downloads the Galaxy S26 NPU path:

pip install -U huggingface_hub
hf download litert-community/Phonon-2 --local-dir Phonon-2 --include phonon2_encoder_all_f16_sm8850.tflite \
  --include "phonon2_mel_*.tflite" --include phonon2_decoder_i8.tflite --include tokenizer.json \
  --include "*.cc" --include "*.sh" --include "*.py"
cd Phonon-2

For another path, replace the first --include with "phonon2_encoder_*s_f16.tflite" (the first release's NPU files, compiled on the phone), phonon2_encoder_all_f16.tflite (GPU), phonon2_encoder_all_i8.tflite (CPU) or "phonon2_encoder_*s_i8.tflite" (the first release's CPU files, which the desktop script below reads). Without any --include it downloads every file.

2. Your audio

The desktop script and the Android runners below read 16 kHz mono 16-bit WAV, and the Android runners read a folder with an index file. The block below writes those WAVs and the index from any format ffmpeg reads; replace sample1.wav sample2.wav with your files.

mkdir -p audio && : > audio/index.tsv && i=0
for f in sample1.wav sample2.wav; do
  id=$(printf %03d $i)
  ffmpeg -loglevel error -y -i "$f" -ac 1 -ar 16000 -c:a pcm_s16le "audio/$id.wav"
  printf '%s\t%s\t%s.wav\n' "$id" "$(basename "$f")" "$id" >> audio/index.tsv
  i=$((i + 1))
done

The accuracy tables below score LibriSpeech's 16-bit files and VoxPopuli's files as published (32-bit float containers holding 16-bit values), whose samples this block keeps; a 32-bit float source with finer values is rounded to 16-bit PCM here, which can change a transcript by a token.

3a. On a desktop (Python, CPU)

Save this as transcribe.py in the Phonon-2 folder (it reads the first release's three i8 encoders: download them with the "phonon2_encoder_*s_i8.tflite" include above):

import sys
import time
import wave

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from tokenizers import Tokenizer

BUCKETS = (10, 20, 30)  # seconds: each 30 s window goes to the smallest graph that holds it
mel = {b: Interpreter(f"phonon2_mel_{b}s.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
enc = {b: Interpreter(f"phonon2_encoder_{b}s_i8.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
dec = Interpreter("phonon2_decoder_i8.tflite", num_threads=1)
predict, joint = dec.get_signature_runner("predict"), dec.get_signature_runner("joint")
tokenizer = Tokenizer.from_file("tokenizer.json")
BLANK, EOS, WINDOW = 8192, 3, 479840


def read_wav(path):  # 16 kHz mono 16-bit PCM
    with wave.open(path) as w:
        assert (w.getframerate(), w.getnchannels(), w.getsampwidth()) == (16000, 1, 2), path
        return np.frombuffer(w.readframes(w.getnframes()), "<i2").astype(np.float32) / 32768


def transcribe_window(x):
    n = len(x)
    b = next(b for b in BUCKETS if n <= b * 16000)
    audio = np.zeros((1, b * 16000), np.float32)
    audio[0, :n] = x
    m = mel[b](audio=audio, n_valid=np.array([n], np.int32))
    e = enc[b](features=m["features"], mel_mask=m["mel_mask"])["enc"][0]
    frames = int(m["frame_mask"].sum())
    state = predict(token=np.array([[BLANK]], np.int32), h=np.zeros((2, 1, 640), np.float32), c=np.zeros((2, 1, 640), np.float32))
    ids, t, steps, run = [], 0, 0, 0
    while t < frames and steps < 3750:
        steps += 1
        logits = joint(enc_frame=e[t:t + 1], dec_out=state["dec_out"])["logits"][0]
        token, d = int(np.argmax(logits[:8193])), int(np.argmax(logits[8193:]))
        if token == BLANK and d == 0:
            d = 1
        if token != BLANK:
            ids.append(token)
            if token == EOS:
                break
            state = predict(token=np.array([[token]], np.int32), h=state["h_out"], c=state["c_out"])
        if d == 0:
            run += 1
            if run >= 10:
                d = 1
        if d > 0:
            run = 0
        t += d
    return tokenizer.decode(ids, skip_special_tokens=True)


for path in sys.argv[1:]:
    x = read_wav(path)
    t0 = time.perf_counter()
    texts = [transcribe_window(x[s:s + WINDOW]) for s in range(0, len(x), WINDOW) if len(x[s:s + WINDOW]) >= 320]
    print(f"{path}\t{len(x) / 16000:.1f} s audio\t{time.perf_counter() - t0:.2f} s\t" + " ".join(t for t in texts if t))

pip install ai-edge-litert==2.3.0 tokenizers numpy
python transcribe.py audio/*.wav

3b. On an Android phone (native runners)

Build the runners with the Android NDK r29 (we built them on macOS; the path below is Android Studio's default there). The script downloads the LiteRT 2.3.0 C headers (litert_cc_sdk.zip from LiteRT's GitHub release) and the runtime libraries (Google's Maven), and checks their sha256.

NDK=$HOME/Library/Android/sdk/ndk/29.0.13113456 bash build_android_p8.sh

Copy the runners, the mel and decoder files and your audio to the phone:

adb shell mkdir -p /data/local/tmp/Phonon-2
adb push ultra_asr_runner_p8 ultra_asr_runner_p libs/libLiteRt.so phonon2_mel_*.tflite phonon2_decoder_i8.tflite audio /data/local/tmp/Phonon-2/
adb shell chmod 755 /data/local/tmp/Phonon-2/ultra_asr_runner_p8 /data/local/tmp/Phonon-2/ultra_asr_runner_p

Then run one of the four paths below. Each writes out.jsonl, which the block after them turns into text.

NPU, Galaxy S26 (Qualcomm SM8850). The sm8850 file is compiled for that chip, and the phone loads it without compiling. The block pushes LiteRT's Qualcomm libraries (libLiteRtDispatch_Qualcomm.so and libLiteRtCompilerPlugin_Qualcomm.so, from the litert-npu-runtime-qualcomm package that build_android_p8.sh fetches) and the QNN libraries of the Qualcomm AI Runtime SDK (QAIRT) 2.50.0.260828, the version the file was compiled with, which you download from Qualcomm; copy these eight files flat into a folder qnn/: from lib/aarch64-android/, libQnnHtp.so, libQnnHtpPrepare.so, libQnnHtpV81Stub.so, libQnnHtpV81CalculatorStub.so, libQnnSystem.so, libQnnIr.so and libQnnSaver.so; from lib/hexagon-v81/unsigned/, libQnnHtpV81Skel.so. Without libLiteRtClGlAccelerator.so on the phone (the GPU block pushes it) the run prints GPU accelerator could not be loaded and registered on stderr and goes on.

adb push libs/libLiteRtDispatch_Qualcomm.so libs/libLiteRtCompilerPlugin_Qualcomm.so qnn/. phonon2_encoder_all_f16_sm8850.tflite /data/local/tmp/Phonon-2/
adb shell 'cd /data/local/tmp/Phonon-2 && LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH="/data/local/tmp/Phonon-2;/vendor/dsp/cdsp;/vendor/lib/rfsa/adsp;/system/lib/rfsa/adsp;/dsp" \
  ./ultra_asr_runner_p8 --accel npu+cpu --libdir . --enc-all phonon2_encoder_all_f16_sm8850.tflite \
  --mel phonon2_mel_30s.tflite --mel10 phonon2_mel_10s.tflite --mel20 phonon2_mel_20s.tflite --mel5 phonon2_mel_5s.tflite \
  --mel-bucket 7.5=phonon2_mel_7.5s.tflite --mel-bucket 15=phonon2_mel_15s.tflite --mel-bucket 25=phonon2_mel_25s.tflite \
  --dec phonon2_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 2 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

NPU compiled on the phone (the first release's files). The sm8850 file is compiled for that chip only. ultra_asr_runner_p compiles each f16 encoder for the phone's Hexagon NPU the first time (LiteRT's Qualcomm compiler plugin, on-device JIT) and keeps the result in --cache-dir; we have run this path on the Galaxy S26 only, with the libraries of the block above. Download its encoders with their own --include (above): the push takes every phonon2_encoder_*_f16.tflite in the folder, and phonon2_encoder_all_f16.tflite matches it too. --window-samples is the window each graph holds: 479,840 samples for the 30 s graph, 320,000 and 160,000 for the others. Compile one encoder per process (one compile alone takes the runner to 4.8–6.3 GB, see Notes): the loop stops at the first encoder that does not compile, and the last command runs all three from the cache only when all three compiled. Run this block before the GPU block, with other apps closed:

adb push libs/libLiteRtDispatch_Qualcomm.so libs/libLiteRtCompilerPlugin_Qualcomm.so qnn/. phonon2_encoder_*_f16.tflite /data/local/tmp/Phonon-2/
NPU_ENV='LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH="/data/local/tmp/Phonon-2;/vendor/dsp/cdsp;/vendor/lib/rfsa/adsp;/system/lib/rfsa/adsp;/dsp"'
compiled=0
for b in 30 20 10; do
  adb shell "cd /data/local/tmp/Phonon-2 && $NPU_ENV ./ultra_asr_runner_p --accel npu+cpu --libdir . --cache-dir cache \
    --model phonon2_encoder_${b}s_f16.tflite --mel phonon2_mel_${b}s.tflite --window-samples $((b * 16000 - (b / 30) * 160)) \
    --dec phonon2_decoder_i8.tflite --fixtures audio --count 1 --rounds 0 --dump none --out out_compile" > compile_$b.log &&
    grep -q '^done wall_ms=' compile_$b.log || { echo "the ${b} s encoder did not compile (see compile_$b.log): stopped" >&2; break; }
  compiled=$((compiled + 1))
done
[ "$compiled" = 3 ] && adb shell "cd /data/local/tmp/Phonon-2 && $NPU_ENV ./ultra_asr_runner_p --accel npu+cpu --libdir . --cache-dir cache \
  --model phonon2_encoder_30s_f16.tflite --mel phonon2_mel_30s.tflite \
  --model10 phonon2_encoder_10s_f16.tflite --mel10 phonon2_mel_10s.tflite \
  --model20 phonon2_encoder_20s_f16.tflite --mel20 phonon2_mel_20s.tflite \
  --dec phonon2_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 1 \
  --fixtures audio --count \$(grep -c . audio/index.tsv) --rounds 0 --dump none --out out" > out.jsonl

GPU. One file for every window length, with --gpu-share-constants 1:

adb push libs/libLiteRtClGlAccelerator.so phonon2_encoder_all_f16.tflite /data/local/tmp/Phonon-2/
adb shell 'cd /data/local/tmp/Phonon-2 && LD_LIBRARY_PATH=. ./ultra_asr_runner_p8 --accel gpu --gpu-precision fp16 --gpu-share-constants 1 --libdir . \
  --enc-all phonon2_encoder_all_f16.tflite \
  --mel phonon2_mel_30s.tflite --mel10 phonon2_mel_10s.tflite --mel20 phonon2_mel_20s.tflite --mel5 phonon2_mel_5s.tflite \
  --mel-bucket 7.5=phonon2_mel_7.5s.tflite --mel-bucket 15=phonon2_mel_15s.tflite --mel-bucket 25=phonon2_mel_25s.tflite \
  --dec phonon2_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 2 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

CPU. One i8 file for every window length. --threads 6 is what we ran on the Galaxy S26; set it for your phone (the Pixel 11 Pro has 7 CPU cores, and its row in the performance table used 5). Without libLiteRtClGlAccelerator.so on the phone the run prints GPU accelerator could not be loaded and registered on stderr and goes on.

adb push phonon2_encoder_all_i8.tflite /data/local/tmp/Phonon-2/
adb shell 'cd /data/local/tmp/Phonon-2 && LD_LIBRARY_PATH=. ./ultra_asr_runner_p8 --accel cpu --libdir . \
  --enc-all phonon2_encoder_all_i8.tflite \
  --mel phonon2_mel_30s.tflite --mel10 phonon2_mel_10s.tflite --mel20 phonon2_mel_20s.tflite --mel5 phonon2_mel_5s.tflite \
  --mel-bucket 7.5=phonon2_mel_7.5s.tflite --mel-bucket 15=phonon2_mel_15s.tflite --mel-bucket 25=phonon2_mel_25s.tflite \
  --dec phonon2_decoder_i8.tflite --threads 6 --mel-threads 1 --dec-threads 1 --pipeline 2 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

Text.

pip install tokenizers
python tdt_text.py tokenizer.json out.jsonl

Each line of out.jsonl that starts with {"kind":"utt" holds one file's token ids (tok), their encoder frame (frame = window index × 375 + the frame within the window) and duration (dur), and the stage times in ms. The burst_summary line gives rtf_wall, the run's wall clock from the first window's start to the last window's end over the audio time (the model load and the warm-up file excluded), and rtf_burst, the summed stage times over the audio time, which counts the overlapped time twice under --pipeline 1 and --pipeline 2.

To remove everything from the phone: adb shell rm -rf /data/local/tmp/Phonon-2.

Accuracy

Word error rate (%) of the NPU form on a Galaxy S26 (phonon2_encoder_all_f16_sm8850 on the NPU, the int8 decoder on the CPU), against the PyTorch reference. The reference is transformers 5.20.2 ParakeetForTDT in float32 with the weights read out of Phonon-2's container (every key loaded, none missing or unexpected), the same 30 s windows with a valid-length mask, and the same greedy TDT decoding. Every utterance of each set. The first pair of WER columns is scored as the Open ASR Leaderboard scores (its English normalizer, compound merging, kaldialign), the (jiwer) pair is plain corpus WER (jiwer) on the normalized text, without the compound merging.

set	utterances	WER, Galaxy S26 NPU	WER, PyTorch reference	WER (jiwer), Galaxy S26 NPU	WER (jiwer), PyTorch reference
LibriSpeech test-clean	2,620	1.726	1.736	2.149	2.153
LibriSpeech test-other	2,939	3.921	3.938	4.355	4.364
VoxPopuli voxpopuli_cleaned_aa (human-corrected references)	628	2.452	2.458	2.677	2.671
VoxPopuli voxpopuli (references from parliamentary records)	1,842	6.880	6.905	7.166	7.191

Sets and revision: hf-audio/open-asr-leaderboard @ de79e57a. The first release's three f16 files, compiled on the same phone, gave 1.728 and 2.151 on LibriSpeech test-clean and the same values as the sm8850 file on the other three sets.

Against Phonon-2's card. Run on whole utterances instead of 30 s windows, our PyTorch reference gives 1.726 on LibriSpeech test-clean, 3.916 on test-other and 2.458 on voxpopuli_cleaned_aa, where Phonon-2's card lists 1.72, 3.92 and 2.46 (2.46 under the name VoxPopuli; on the voxpopuli split the same run gives 6.896, and the leaderboard's own script notes that split's references contain spans that were not spoken). The 628 clips of voxpopuli_cleaned_aa are a subset of the 1,842 with the same audio bytes; their references differ on 626.

Every form on the 655-utterance LibriSpeech subset of the performance table, same scoring (reference 1.769 / 2.173):

phone, backend	WER	WER (jiwer)
Galaxy S26, NPU (sm8850 file)	1.776	2.203
Galaxy S26, NPU (three f16 files compiled on the phone)	1.784	2.211
Galaxy S26, GPU (one f16 file)	1.784	2.203
Galaxy S26, GPU (three f16 files)	1.799	2.218
Galaxy S26, CPU (one i8 file)	1.792	2.211
Galaxy S26, CPU (three i8 files)	1.792	2.211
Pixel 11 Pro, GPU (one f16 file)	1.769	2.173
Pixel 11 Pro, CPU (one i8 file)	1.784	2.158
Pixel 11 Pro, CPU (three i8 files)	1.784	2.158

Performance

phone	backend	RTF	× real time	peak memory	encoder, one 30 s window	decoding, one 30 s window
Galaxy S26	NPU (Hexagon, sm8850 file, --pipeline 2)	0.005616	178×	1.63 GB (1,588,412 kB)	177.6 ms	19.2 ms
Galaxy S26	NPU (Hexagon, three f16 files compiled on the phone, --pipeline 1)	0.009210	109×	4.80 GB (4,688,840 kB)	—	—
Galaxy S26	GPU (Adreno, one f16 file, fp16 arithmetic, --pipeline 2)	0.023113	43×	3.64 GB (3,556,088 kB)	—	—
Galaxy S26	GPU (Adreno, three f16 files, fp16 arithmetic, --pipeline 0)	0.025732	39×	7.37 GB (7,193,436 kB)	—	—
Galaxy S26	CPU (one i8 file, 6 threads, --pipeline 2)	0.016822	59×	6.18 GB (6,035,236 kB)	—	—
Galaxy S26	CPU (three i8 files, 8 threads, --pipeline 0)	0.022280	45×	4.51 GB (4,406,376 kB)	—	—
Pixel 11 Pro	GPU (PowerVR, one f16 file, fp16 arithmetic, --pipeline 2)	0.038671	26×	2.41 GB (2,356,344 kB)	491.1 ms	53.9 ms
Pixel 11 Pro	CPU (one i8 file, 5 threads, --pipeline 2)	0.055918	18×	6.16 GB (6,014,964 kB)	—	—
Pixel 11 Pro	CPU (three i8 files, 5 threads, --pipeline 0)	0.087680	11×	4.50 GB (4,389,672 kB)	—	—

• RTF = rtf_wall: the wall clock of a run from the first window's start to the last window's end over the audio time, the model load and one warm-up utterance excluded, for 655 LibriSpeech test-clean utterances (4,875.8 s of audio) run one after another in one process with the settings of the row (plus a per-window dump of the outputs for scoring; the mel graphs at 2 threads, 1 on the one-file CPU rows, the decoder at 1). The one-file rows ran the seven window lengths, the windows going 279 / 141 / 84 / 95 / 36 / 15 / 8 to the 5 / 7.5 / 10 / 15 / 20 / 25 / 30 s graphs, with ultra_asr_runner_p7, the build before ultra_asr_runner_p8 (the Galaxy S26's one-file GPU row with ultra_asr_runner_p8). The three-file rows are the first release's: ultra_asr_runner_p, the windows going 504 / 131 / 23 to the 10 / 20 / 30 s graphs. The two NPU rows and the Pixel 11 Pro's GPU row are the mean of two runs, the other rows one run each.
• --pipeline 1 against --pipeline 0 on the NPU with the three f16 files compiled on the phone (RTF): 0.009210 against 0.011520 (one session, in the order 0, 1, 1, 0).
• ultra_asr_runner_p8 is ultra_asr_runner_p7 (not in this repository) with options the blocks above do not use (deeper queues, a second encoder chosen by the window length); on our 82-utterance test subset (the two runners built for macOS, on the CPU with the one-file i8 encoder, the seven window lengths, --pipeline 2) it gave ultra_asr_runner_p7's tokens, frames and durations on all 83 windows.
• Per window (the last two columns): one 30 s window holding a 24.5 s utterance, the median of 20 calls in one process started on a rested phone; the decoding column is the whole TDT loop of that window on the CPU. The CPU clocks did not stay uncapped through the calls: on the Galaxy S26 they were capped during 4 of the 20 encoder calls and all 20 decoding calls, on the Pixel 11 Pro during 15 of the 20 encoder calls, with its decoding calls uncapped. The shorter graphs of the sm8850 file, on cuts of the same utterance (each with the CPU clocks capped during all 20 calls): 93.6 / 81.2 / 47.9 / 42.9 / 33.7 / 29.9 ms for 25 / 20 / 15 / 10 / 7.5 / 5 s. The Pixel 11 Pro's per-window figures, the table's 491.1 and 53.9 ms included, come from a four-graph build of the same f16 graphs (5, 10, 20, 30 s; not in this repository); its shorter graphs: 364.4 / 275.8 / 240.4 ms for 20 / 10 / 5 s (capped during 16, 0 and 20 of the 20 calls). "—": not measured.
• State of the phone. The runs did not start from the same state. The sm8850 runs started at thermal status 0 and 1 with the CPU clocks already capped (2.50 / 2.44 GHz and 2.11 / 2.11 GHz), the Galaxy S26's one-file CPU run at status 0 with no cap; the Pixel 11 Pro's GPU runs started at status 2 and 3, its one-file CPU run at status 0 with no cap. The Galaxy S26's one-file GPU run started at thermal status 3 with the CPU clocks capped at 3.19 / 3.40 GHz, the GPU's clock ceiling at 826 MHz. For the first release's rows: on the Galaxy S26's NPU the first --pipeline 0 run (thermal status 0 at its start) took 0.011097 and the last (status 2) 0.011942. The Galaxy S26's CPU run started at status 0 with no CPU frequency cap and ended with the CPU clocks capped at 1.79 / 1.75 GHz. Its GPU run came last in its session: status 2, the GPU's clock ceiling at 646 MHz at the start and 500 MHz at the end (1,300 MHz at rest), the CPU clocks capped at 1.79 / 1.75 GHz at the start. The Pixel 11 Pro's CPU run started at status 1 and ended with the CPU clocks capped at 1.92 / 1.80 / 2.00 GHz.
• Peak memory: the runner's VmHWM during the 655-utterance run (GB = kB × 1024 / 10⁹; of a row's two runs, the larger).
• Galaxy S26 (SM-S942Q, Android 16) and Pixel 11 Pro (Tensor G6, Android 17); LiteRT 2.3.0 from Google's Maven; on the NPU the QNN libraries of QAIRT 2.50.0.260828 for Hexagon v81; 2026-11-08.

Notes

• First NPU run, sm8850 file. The phone compiles nothing: LiteRT logs that the model is pre-compiled and applies no compiler plugin, and on a Galaxy S26 the runner loaded the file's seven graphs in 0.84–0.96 s (three runs), each graph on the NPU as one partition.
• First NPU run, the first release's f16 files. On a Galaxy S26 the on-device compile took 78–84 s for the 30 s and 20 s encoders and 176–209 s for the 10 s encoder (one encoder per process), and each compiled encoder takes 1.2 GB in the cache folder; a later run loads each of the three from the cache in 1.0–3.2 s. While one encoder compiles, the runner's peak memory is 4.8–6.3 GB, and the phone's available memory went down to 1.28 GB. Start the first NPU run fresh, with other apps closed.
• GPU start-up. Each GPU run first prepares the encoder for the GPU (the blocks above keep no GPU program cache): loading the one f16 file took 37.8 and 38.5 s on the Pixel 11 Pro in the performance table's two runs, started at thermal status 3 and 2, and 11.7 s in a later run started at status 0; 14.2 s on the Galaxy S26 (one run, started at status 3).
• GPU and fp16. In Phonon-2's encoder the squared deviation inside the first LayerNorm reaches 59,451,440 over the 91 windows of our range check, beyond float16's largest value 65,504. The f16 encoders scale each of the 120 LayerNorm inputs by 2^-k (k = 3–8 per norm) and its epsilon by 4^-k, which leaves the float32 result bit-identical and keeps every square in range (1,014.5 at most over the same windows, and 1,262.3 at most over the 1,229 windows of the same check on the 5, 7.5, 15 and 25 s graphs); the GPU (float16 arithmetic) and the NPU run these files.
• int8 decoder. The decoder's weights are int8, one scale per output channel, with int8 activations computed at run time (dynamic range), as in the i8 encoders. Against the float32 decoder (Mac CPU, float32 encoders) it gives the same token sequence on 82 of the 83 windows of our 82-utterance test subset.
• int8 encoders. The i8 encoders round the five-value weights to int8 per output channel (each matrix's median relative error is 0.17 % at the median of the 240 matrices; 0.444 % of the small values round to zero), and the first release's three keep their WER on a Mac CPU within 0.023 points of the reference on the four sets above. Their depthwise convolutions stay float32 so that XNNPACK runs them. On both phones the one i8 file gave the same tokens, frames and durations as the seven graphs in seven files, on all 658 windows of the 655-utterance runs.
• Converted from the weights Phonon-2's fermion_container.py format holds, loaded into transformers 5.20.2's ParakeetForTDT (a five-value matrix becomes float32 with the same values), rewritten as plain PyTorch graphs (masks as float arithmetic, the LSTM step as explicit gates) and exported with litert-torch 0.9.4; the steps and scripts for every .tflite file here are in hf-to-litertlm (REPRODUCE.md, Phonon-2, with its update of 2026-11-08 for the files added after the first release), run as printed from a clean checkout on 2026-11-08 with every one of those .tflite files coming out byte for byte the same.

License

The model files are under CC-BY-4.0, as FermionResearch/Phonon-2 and NVIDIA's parakeet-tdt-0.6b-v3 are. Weights © Fermion Research, derived from Parakeet TDT 0.6B v3, © NVIDIA Corporation. Changes made here: the weights were expanded to float16 and int8 as described above, and the graphs were rewritten for LiteRT.

Phonon-2's NOTICE says its training audio includes CHiME-6, distributed under CC-BY-SA-4.0 (https://chimechallenge.github.io/chime6/), and SPGISpeech, used under Kensho's public terms (https://datasets.kensho.com/datasets/spgispeech). Read that NOTICE before you redistribute: https://huggingface.co/FermionResearch/Phonon-2/blob/main/NOTICE.

The runners and the scripts (ultra_asr_runner_p8.cc, build_android_p8.sh, ultra_asr_runner_p.cc, build_android.sh, tdt_text.py) are under the Apache License 2.0.