Haas-ja-v1.0 for LiteRT

This repository is a conversion of third-intelligence/Haas-ja-v1.0 (Japanese speech-to-text) to LiteRT, made with litert-torch 0.9.4 and ai-edge-quantizer 0.10.0 for LiteRT 2.3.0: on a Galaxy S26 its float16 encoders on the NPU transcribe 134× faster than real time, 63× on the GPU, and its int8 encoders 46× on the CPU (the 650 utterances of the FLEURS Japanese test set, one after another, measured 2026-11-08; the NPU run with the host pipeline on, the GPU and CPU runs serial and on a warm phone, see Performance). On those 650 utterances the character error rate (CER strict, defined under Accuracy) on the Galaxy S26's NPU is 3.7009 %, against 3.6886 % for our float32 PyTorch run of the same audio. Speed is still being tuned (updates follow).

third-intelligence/Haas-ja-v1.0 follows NVIDIA's Parakeet-TDT: a FastConformer encoder (24 layers, hidden size 1,024) and a Token-and-Duration Transducer (TDT) decoder whose prediction network is two layers of Kimi Delta Attention instead of an LSTM (its card). The checkpoint (Haas-ja-v1.0.nemo) stores its weights in bfloat16. The encoder graphs are built by the same code as litert-community/parakeet-ultra's from Haas-ja-v1.0's weights (with Haas-ja-v1.0's encoder input scale of 32 folded into the subsampling's linear layer); the prediction network is a new graph that carries the attention state between steps, and the Android runner is parakeet-ultra's with that state handled generically. The speeds on this card are for this repository's .tflite files run with LiteRT 2.3.0's C API, not for LiteRT-LM or for any other conversion of this model.

Files

file	bytes	what	backend
haas_ja_encoder_10s_i8.tflite	616,306,528	encoder for windows up to 10 s; int8 weights, one scale per output channel, in the linear layers and the two 1×1 convolutions of the subsampling, with int8 activations computed at run time; the depthwise convolutions and the 24 relative-position tables stay float32	CPU
haas_ja_encoder_20s_i8.tflite	640,882,528	the same, up to 20 s	CPU
haas_ja_encoder_30s_i8.tflite	665,458,528	the same, up to 30 s	CPU
haas_ja_encoder_10s_f16.tflite	1,194,675,168	encoder for windows up to 10 s; float16 weights in the linear and convolution layers, LayerNorm inputs rescaled so that float16 arithmetic does not overflow	GPU, NPU
haas_ja_encoder_20s_f16.tflite	1,219,251,168	the same, up to 20 s	GPU, NPU
haas_ja_encoder_30s_f16.tflite	1,243,827,168	the same, up to 30 s	GPU, NPU
haas_ja_mel_10s.tflite	1,608,068	log-mel front end, 10 s of 16 kHz audio, with the checkpoint's own window and mel filterbank values	CPU
haas_ja_mel_20s.tflite	2,252,068	the same, 20 s	CPU
haas_ja_mel_30s.tflite	2,896,068	the same, 30 s	CPU
haas_ja_decoder_i8.tflite	6,479,920	TDT prediction network (predict, two Kimi Delta Attention layers) and joint network (joint), int8 weights	CPU
tokenizer.model	279,711	the original's SentencePiece tokenizer (the .nemo member 74b21e79e00d473eb171526fc011af5b_tokenizer.model, byte for byte)	host
haas_asr_runner.cc, build_android.sh		a command-line runner for Android (LiteRT C API) and its build script: parakeet-ultra's runner (ultra_asr_runner_p.cc) with the decoder state read from the predict signature, the vocabulary size read from joint, no end token and 30 s windows of 480,000 samples (the runner reads the model files from its arguments)
tdt_text.py		turns the runner's token ids into text with tokenizer.model	host

Each encoder file holds the whole encoder; the three window lengths only save the work of padding short audio. Take the i8 set for the CPU (1.94 GB with the mel and decoder files) or the f16 set for the GPU or the NPU (3.67 GB with them).

How it works

audio (16 kHz mono) → 30 s windows → mel front end → encoder → greedy TDT decoding → text

• Windows. The audio is cut into consecutive windows of 480,000 samples (30 s); the last one is shorter. A window under 320 samples (two mel frames) is dropped and has no text. Each window is transcribed on its own, and the window texts are joined with no separator. Long audio is cut at fixed 30 s points, not at pauses (the original's Haas-ASR code, transcribe_longform, cuts at quiet points); we did not measure what happens to a word that crosses a cut. No utterance of the FLEURS test set is longer than 30 s (the longest is 28.2 s), so the accuracy below is for single windows.
• Buckets. Each window goes to the smallest graph pair that holds it: up to 160,000 samples → the 10 s mel and encoder, up to 320,000 → 20 s, else 30 s. The shorter graphs only skip the work on padding: in float32 they gave the same tokens as our PyTorch reference on all 650 FLEURS utterances (131 / 489 / 30 of them on the 10 / 20 / 30 s graphs).
• Mel front end (haas_ja_mel_*, inputs audio [1, samples] float32 and n_valid [1] int32): NeMo's AudioToMelSpectrogramPreprocessor as this checkpoint configures it — pre-emphasis 0.97, STFT (512 points, 400-sample window, hop 160), 128 mel bins, log with a 2^-24 guard, per-bin normalization over the valid frames. The window and the filterbank are the values stored in the checkpoint (bfloat16 values: they differ from a computed Hann window by up to 1.94e-3 and from librosa's Slaney filterbank by up to 1.17e-4). Outputs features [1, 128, T], mel_mask [1, T] and frame_mask [1, T / 8] (T = 1,000 / 2,000 / 3,000).
• Encoder (haas_ja_encoder_*, inputs features and mel_mask as the mel graph gives them): output enc [1, T / 8, 640], the FastConformer output already projected by the joint network's encoder layer (1,024 → 640); rows past the valid frames are undefined.
• Decoding (on the host, haas_ja_decoder_i8): predict (token [1, 1] int32 and four state tensors: kda0_rec, kda1_rec [1, 5, 64, 64] and kda0_conv, kda1_conv [1, 960, 4], float32) → dec_out [1, 640] and the new state as kda0_rec_out and so on; joint (one enc row [1, 640], dec_out) → 3,078 logits = 3,073 token logits (3072 = blank) + 5 duration logits (0–4 frames). Start from blank with zero state; at each step take the arg-max token and duration; a blank with duration 0 moves one frame; a non-blank token is emitted and fed to predict; after ten tokens emitted on one frame the decoder moves one frame; stop at the last valid frame (the sum of frame_mask). There is no end token and no step limit (NeMo 2.7.3's greedy TDT decoding).
• Text. Turn each window's token ids into SentencePiece pieces with tokenizer.model and decode them (decode_pieces; byte-fallback pieces become their bytes), then remove a space that comes before a punctuation mark of the model's vocabulary, as NeMo does; join the windows and strip the result (tdt_text.py).

Use it

1. Download

pip install -U huggingface_hub
hf download litert-community/Haas-ja-v1.0 --local-dir Haas-ja-v1.0
cd Haas-ja-v1.0

2. Your audio

Both runners below (the Python script of 3a and the Android runner of 3b) read 16 kHz mono 16-bit WAV, and the Android runner reads a folder with an index file. The block below writes those WAVs and the index from any format ffmpeg reads; replace sample1.wav sample2.wav with your files.

mkdir -p audio && : > audio/index.tsv && i=0
for f in sample1.wav sample2.wav; do
  id=$(printf %03d $i)
  ffmpeg -loglevel error -y -i "$f" -ac 1 -ar 16000 -c:a pcm_s16le "audio/$id.wav"
  printf '%s\t%s\t%s.wav\n' "$id" "$(basename "$f")" "$id" >> audio/index.tsv
  i=$((i + 1))
done

The accuracy and speed tables below use the FLEURS files as asr-ja-bench rebuilds them from google/fleurs (16-bit PCM, 16 kHz); the block above writes their samples unchanged. For a 32-bit float file it rounds the samples to 16-bit PCM, which can change a transcript by a token (seen on another model's conversion; not measured on this one).

3a. On a desktop (Python, CPU)

Save this as transcribe.py in the Haas-ja-v1.0 folder:

import sys
import time
import wave

import numpy as np
from ai_edge_litert.interpreter import Interpreter

from tdt_text import Text

BUCKETS = (10, 20, 30)  # seconds: each 30 s window goes to the smallest graph that holds it
mel = {b: Interpreter(f"haas_ja_mel_{b}s.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
enc = {b: Interpreter(f"haas_ja_encoder_{b}s_i8.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
dec = Interpreter("haas_ja_decoder_i8.tflite", num_threads=1)
predict, joint = dec.get_signature_runner("predict"), dec.get_signature_runner("joint")
STATE = {"kda0_rec": (1, 5, 64, 64), "kda0_conv": (1, 960, 4), "kda1_rec": (1, 5, 64, 64), "kda1_conv": (1, 960, 4)}
text = Text("tokenizer.model")
BLANK, WINDOW = 3072, 480000


def read_wav(path):  # 16 kHz mono 16-bit PCM
    with wave.open(path) as w:
        assert (w.getframerate(), w.getnchannels(), w.getsampwidth()) == (16000, 1, 2), path
        return np.frombuffer(w.readframes(w.getnframes()), "<i2").astype(np.float32) / 32768


def transcribe_window(x):
    n = len(x)
    b = next(b for b in BUCKETS if n <= b * 16000)
    audio = np.zeros((1, b * 16000), np.float32)
    audio[0, :n] = x
    m = mel[b](audio=audio, n_valid=np.array([n], np.int32))
    e = enc[b](features=m["features"], mel_mask=m["mel_mask"])["enc"][0]
    frames = int(m["frame_mask"].sum())
    out = predict(token=np.array([[BLANK]], np.int32), **{k: np.zeros(s, np.float32) for k, s in STATE.items()})
    ids, t, run = [], 0, 0
    while t < frames:
        logits = joint(enc_frame=e[t:t + 1], dec_out=out["dec_out"])["logits"][0]
        token, d = int(np.argmax(logits[:BLANK + 1])), int(np.argmax(logits[BLANK + 1:]))
        if token == BLANK and d == 0:
            d = 1
        if token != BLANK:
            ids.append(token)
            out = predict(token=np.array([[token]], np.int32), **{k: out[k + "_out"] for k in STATE})
        if d == 0:
            run += 1
            if run >= 10:
                d = 1
        if d > 0:
            run = 0
        t += d
    return ids


for path in sys.argv[1:]:
    x = read_wav(path)
    t0 = time.perf_counter()
    wins = [transcribe_window(x[s:s + WINDOW]) for s in range(0, len(x), WINDOW) if len(x[s:s + WINDOW]) >= 320]
    print(f"{path}\t{len(x) / 16000:.1f} s audio\t{time.perf_counter() - t0:.2f} s\t" + text.windows(wins))

pip install ai-edge-litert==2.3.0 sentencepiece numpy
python transcribe.py audio/*.wav

3b. On an Android phone (native runner)

Build the runner with the Android NDK r29; the script needs curl, unzip and shasum or sha256sum (we built it on macOS; the path below is Android Studio's default there). The script downloads the LiteRT 2.3.0 C headers (litert_cc_sdk.zip from LiteRT's GitHub release) and the runtime libraries (Google's Maven), and checks their sha256.

NDK=$HOME/Library/Android/sdk/ndk/29.0.13113456 bash build_android.sh

Copy the runner, the CPU files and your audio to the phone, run it, and turn its output into text:

adb shell mkdir -p /data/local/tmp/haas-ja
adb push haas_asr_runner libs/libLiteRt.so haas_ja_mel_*.tflite haas_ja_encoder_*_i8.tflite haas_ja_decoder_i8.tflite audio /data/local/tmp/haas-ja/
adb shell chmod 755 /data/local/tmp/haas-ja/haas_asr_runner

adb shell 'cd /data/local/tmp/haas-ja && LD_LIBRARY_PATH=. ./haas_asr_runner --accel cpu --libdir . \
  --model haas_ja_encoder_30s_i8.tflite --mel haas_ja_mel_30s.tflite \
  --model10 haas_ja_encoder_10s_i8.tflite --mel10 haas_ja_mel_10s.tflite \
  --model20 haas_ja_encoder_20s_i8.tflite --mel20 haas_ja_mel_20s.tflite \
  --dec haas_ja_decoder_i8.tflite --threads 8 --mel-threads 2 --dec-threads 1 --pipeline 1 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

pip install sentencepiece
python tdt_text.py tokenizer.model out.jsonl

Each line of out.jsonl that starts with {"kind":"utt" holds one file's token ids (tok), their encoder frame (frame = window index × 375 + the frame within the window; 375 = the encoder frames of a 30 s window) and duration (dur), and the stage times in ms. The burst_summary line gives rtf_wall, the run's wall clock from the first window's start to the last window's end over the audio time (the model load and the warm-up file excluded), and rtf_burst, the summed stage times over the audio time, which counts the overlapped time twice under --pipeline 1. On the Galaxy S26's NPU --pipeline 1 was faster: rtf_wall 0.007489 against 0.008936 with --pipeline 0 (two runs each, in the order 0, 1, 1, 0); we measured the GPU and the CPU with --pipeline 0 only. The CPU run prints GPU accelerator could not be loaded and registered on stderr: the GPU library is not on the phone for this run, and the run continues on the CPU.

NPU (Galaxy S26, Qualcomm Hexagon). The runner compiles each f16 encoder for the Hexagon NPU on the phone the first time (LiteRT's Qualcomm compiler plugin, on-device JIT) and keeps the result in --cache-dir. It needs LiteRT's Qualcomm plugin libraries (libLiteRtDispatch_Qualcomm.so and libLiteRtCompilerPlugin_Qualcomm.so, from the litert-npu-runtime-qualcomm package that build_android.sh fetches) and the QNN libraries of the Qualcomm AI Runtime SDK (QAIRT) 2.50.0.260828, which you download from Qualcomm; copy these eight files flat into a folder qnn/: from lib/aarch64-android/, libQnnHtp.so, libQnnHtpPrepare.so, libQnnHtpV81Stub.so, libQnnHtpV81CalculatorStub.so, libQnnSystem.so, libQnnIr.so and libQnnSaver.so; from lib/hexagon-v81/unsigned/, libQnnHtpV81Skel.so. These V81 libraries are for the Galaxy S26's Hexagon NPU; we have not run this on other Snapdragon phones. --window-samples is the window each graph holds: 480,000 samples for the 30 s graph, 320,000 and 160,000 for the others. Compile one encoder per process (on a Galaxy S26 a process that compiled one encoder peaked at 5.4–6.3 GB of memory, see Notes): the loop stops at the first encoder that does not compile, and the last command runs all three from the cache only when all three compiled. Run this block with other apps closed and before any GPU run (the compile needs up to 6.3 GB, see Notes):

adb push libs/libLiteRtDispatch_Qualcomm.so libs/libLiteRtCompilerPlugin_Qualcomm.so qnn/. haas_ja_encoder_*_f16.tflite /data/local/tmp/haas-ja/
NPU_ENV='LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH="/data/local/tmp/haas-ja;/vendor/dsp/cdsp;/vendor/lib/rfsa/adsp;/system/lib/rfsa/adsp;/dsp"'
compiled=0
for b in 30 20 10; do
  adb shell "cd /data/local/tmp/haas-ja && $NPU_ENV ./haas_asr_runner --accel npu+cpu --libdir . --cache-dir cache \
    --model haas_ja_encoder_${b}s_f16.tflite --mel haas_ja_mel_${b}s.tflite --window-samples $((b * 16000)) \
    --dec haas_ja_decoder_i8.tflite --fixtures audio --count 1 --rounds 0 --dump none --out out_compile" > compile_$b.log &&
    grep -q '^done wall_ms=' compile_$b.log || { echo "the ${b} s encoder did not compile (see compile_$b.log): stopped" >&2; break; }
  compiled=$((compiled + 1))
done
[ "$compiled" = 3 ] && adb shell "cd /data/local/tmp/haas-ja && $NPU_ENV ./haas_asr_runner --accel npu+cpu --libdir . --cache-dir cache \
  --model haas_ja_encoder_30s_f16.tflite --mel haas_ja_mel_30s.tflite \
  --model10 haas_ja_encoder_10s_f16.tflite --mel10 haas_ja_mel_10s.tflite \
  --model20 haas_ja_encoder_20s_f16.tflite --mel20 haas_ja_mel_20s.tflite \
  --dec haas_ja_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 1 \
  --fixtures audio --count \$(grep -c . audio/index.tsv) --rounds 0 --dump none --out out" > out.jsonl

GPU. Push libs/libLiteRtClGlAccelerator.so and the f16 encoders, and run the CPU command with --accel gpu --gpu-precision fp16 and the haas_ja_encoder_*_f16.tflite files, without --threads 8 (the encoder no longer runs on the CPU):

adb push libs/libLiteRtClGlAccelerator.so haas_ja_encoder_*_f16.tflite /data/local/tmp/haas-ja/
adb shell 'cd /data/local/tmp/haas-ja && LD_LIBRARY_PATH=. ./haas_asr_runner --accel gpu --gpu-precision fp16 --libdir . \
  --model haas_ja_encoder_30s_f16.tflite --mel haas_ja_mel_30s.tflite \
  --model10 haas_ja_encoder_10s_f16.tflite --mel10 haas_ja_mel_10s.tflite \
  --model20 haas_ja_encoder_20s_f16.tflite --mel20 haas_ja_mel_20s.tflite \
  --dec haas_ja_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 1 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

To remove everything from the phone: adb shell rm -rf /data/local/tmp/haas-ja.

Accuracy

Character error rate (%) on the 650 utterances of the FLEURS Japanese test set as asr-ja-bench v1.1 rebuilds it (two reference texts corrected by the benchmark), scored with jacer 1.2 (metrics v1.2) as the original's card scores: CER strict (the primary metric), CER flex (kanji / kana choice ignored) and CER kana (readings). The reference is our float32 PyTorch run of a reimplementation of the original's modules (the NeMo 2.7.3 preprocessor arithmetic in PyTorch and the FastConformer encoder as transformers 5.20.2's ParakeetEncoder, both checked against NeMo's own modules on a Mac's CPU, and the Kimi Delta Attention prediction network rewritten in plain PyTorch from flash-linear-attention 0.5.2, whose Triton kernels do not run on macOS) on the same audio, one 30 s window per utterance, greedy TDT decoding; the original's card reports 3.67 / 2.98 / 2.34 on this set (bfloat16, on a GPU).

backend	CER strict	CER flex	CER kana
PyTorch reference (float32, Mac CPU)	3.6886	3.0182	2.3829
Galaxy S26, NPU (f16 encoders)	3.7009	3.0029	2.3438
Galaxy S26, GPU (f16 encoders, float16 arithmetic)	3.6825	3.0029	2.3737
Galaxy S26, CPU (i8 encoders)	3.6978	3.0091	2.3553

Performance

phone	backend	RTF	× real time	encoder per call, 30 / 20 / 10 s window (ms)	mel, 30 s (ms)	TDT decoding of a 30 s window (ms)	peak memory
Galaxy S26	NPU (Hexagon, f16, --pipeline 1)	0.007489	134×	187.2 / 80.5 / 46.8	3.9	12.4	4.95 GB (4,838,264 kB)
Galaxy S26	GPU (f16, fp16 arithmetic, --pipeline 0)	0.015935	63×	157.4 / 106.6 / 60.7	3.6	8.9	7.73 GB (7,553,244 kB)
Galaxy S26	CPU (i8, 8 threads, --pipeline 0)	0.021774	46×	282.3 / 211.2 / 108.6	4.8	18.6	4.49 GB (4,388,868 kB)

• RTF = rtf_wall: the wall clock of a run from the first window's start to the last window's end over the audio time, the model load and one warm-up utterance excluded, for the 650 FLEURS Japanese test utterances (8,511.1 s of audio) run one after another in one process with haas_asr_runner as in "Use it" (the mel graphs at 2 threads, the decoder at 1). The windows went 131 / 489 / 30 to the 10 / 20 / 30 s graphs. NPU: the mean of two --pipeline 1 runs (0.007303 and 0.007676); the mean of two --pipeline 0 runs in the same session (order 0, 1, 1, 0) was 0.008936. GPU and CPU: one --pipeline 0 run each.
• Heat. The GPU and CPU runs are those of a warm phone: the GPU run started from a phone at rest, but the GPU clock was capped at 578 MHz (of 1,300 MHz, thermal power level 9) in 50 of its 72 readings; the CPU run ran with the clocks of the two CPU clusters capped at 1.56 / 1.50 GHz (of 3.63 / 4.74 GHz) in 52 of its 70 readings (thermal status 1–2).
• Per call: the median of 20 calls after 3 warm-up calls, input write and output read included, on FLEURS Japanese test utterance 9518252661993015549 (28.2 s: its 30 s window with 353 valid encoder frames, and its first 20 s and 10 s), the mel graphs at 2 threads and the decoder at 1, each run started from a phone at rest (thermal status 0, no CPU frequency cap). During the timed calls the phone capped the clocks of the two CPU clusters, in the CPU run down to 2.00 / 1.98 GHz (of 3.63 / 4.74 GHz), so the CPU numbers are those of a capped CPU; the NPU and GPU runs also saw CPU caps during their timed calls (down to 2.90 / 2.88 and 2.75 / 2.67 GHz).
• Peak memory: the runner's VmHWM during the 650-utterance run (GB = kB × 1024 / 10⁹; for the NPU, the larger of its two --pipeline 1 runs).
• Galaxy S26 (SM-S942Q, Android 16); LiteRT 2.3.0 from Google's Maven; on the NPU the QNN libraries of QAIRT 2.50.0.260828 for Hexagon v81; 2026-11-08.

Notes

• First NPU run. On a Galaxy S26 the on-device compile took 56–82 s for the 30 s encoder, 141–235 s for the 10 s encoder and 59–95 s for the 20 s encoder (three runs, one encoder per process), and the three compiled encoders take 3.70 GB in the cache folder (3,611,529 kB); a later run loaded each of the three from the cache in 1.2–2.3 s (four runs). A process that compiled one encoder peaked at 5.4–6.3 GB of resident memory (VmHWM 5,257,204–6,117,668 kB over the nine compiles of the three runs), and the phone's available memory went down to 1.29 GB (1,257,444 kB) in one 20 s encoder's process, the lowest of the nine.
• GPU and fp16. In the checkpoint's encoder the squared deviation inside a LayerNorm reaches 103,163 over the 217 windows of our range check (the first block's output LayerNorm), beyond float16's largest value 65,504. The f16 encoders scale 99 of the 120 LayerNorm inputs by 2^-k (k = 1–4 per norm) and their epsilon by 4^-k, which left the float32 output bit-identical in our check and keeps every square in range (1,023.8 at most over the same windows); the GPU (float16 arithmetic) and the NPU run these files.
• int8 decoder. The decoder's weights are int8, one scale per output channel, and its linear layers take int8 activations computed at run time (dynamic range), as the i8 encoders do. Against the float32 decoder on our reference's encoder output (Mac CPU) it gives the same token sequence on 634 of the 650 FLEURS utterances, and CER 3.6856 / 3.0060 / 2.3668 against 3.6886 / 3.0182 / 2.3829.
• int8 encoders. The i8 encoders are lossy: on a Mac CPU (with the int8 decoder) their CER on the 650 utterances is 3.6978 / 3.0091 / 2.3553 against the reference's 3.6886 / 3.0182 / 2.3829 (CER strict 0.0092 points above it, CER flex and kana below it), and 90 of the 650 transcripts differ from the reference's.

License

The model files are under the Apache License 2.0, as third-intelligence/Haas-ja-v1.0 is; its card says: "The model weights and the inference code are released under the Apache 2.0 License. Third-party components retain their respective licenses." The runner and the scripts (haas_asr_runner.cc, build_android.sh, tdt_text.py) are under the Apache License 2.0.