Qwen3-ASR-1.7B — LiteRT-LM

Qwen/Qwen3-ASR-1.7B-hf converted to the LiteRT-LM (.litertlm) format for on-device speech recognition with Google's LiteRT-LM runtime. Audio in, text out, one file.

Qwen3-ASR-1.7B is Qwen's speech recognition model (2.04 B parameters). A 24-layer audio encoder feeds a 28-layer Qwen3 language model, which writes language <name><asr_text><transcript>. The model card lists language identification and recognition for 30 languages and 22 Chinese dialects; this file was scored on English, Mandarin and Japanese.

The files run on the generic audio path of the released runtime (the 30 s file measured with litert-lm-api 0.17.1, the 15 s file with 0.18.0; the 15 s file also loads and answers on 0.17.1, in the Android sample below). The runtime cuts 16 kHz PCM into 10 ms frames and hands the bundled encoder a window of 30 s or 15 s, depending on the file; the encoder computes the log-mel spectrogram itself. No runtime change is needed.

Galaxy S26 (SM-S942Q), CPU with 4 threads, airplane mode on, LiteRT-LM 0.17.1 through the Kotlin Conversation API (litertlm-android), the 30 s file. The clips are FLEURS test sentence 1698 read in Mandarin, English and Japanese (CC BY 4.0; one linear gain per clip evens out the recording levels). The app plays each clip through the phone's speaker, then sends it in a new conversation. The seconds are the app's wall time of the call, with the language model and audio encoder on the CPU.

File	Recipe	Size
Qwen3-ASR-1.7B-15s.litertlm	LM int8 dynamic (weights) with an int8 embedding table · audio encoder fp16 weights, fp32 compute, 15 s window · prefill signatures for 128 and 256 tokens	2.70 GB
Qwen3-ASR-1.7B.litertlm	LM int8 dynamic (weights) with an int8 embedding table · audio encoder fp16 weights, fp32 compute, 30 s window	2.69 GB

The two files hold the same weights. The 15 s file encodes a clip in 15 s windows (195 audio tokens per window), so a clip of up to 15 s costs half the encoder and prefill work of the 30 s file; its measured response times are in the Performance section. The 30 s file is the one in the screenshots above; the Android sample below runs both files on litertlm-android 0.17.1.

Section	Content	Size
tf_lite_audio_encoder_hw	log-mel front end (the original feature extractor's STFT, Slaney filterbank and normalization), 24-layer encoder, projector	637 MB
tf_lite_prefill_decode	Qwen3 language model, int8, KV cache 1024 tokens	1.74 GB (30 s file), 1.75 GB (15 s file)
tf_lite_embedder	token embedding table, int8	313 MB
HF_Tokenizer_Zlib	tokenizer	2.1 MB
LlmMetadataProto	prompt template, stop tokens, audio settings	0.9 kB

How to send audio

One message holds one clip. The runtime zero-fills the window after the clip, and the encoder turns every window into a fixed number of audio tokens: 390 for the 30 s file, 195 for the 15 s file. Encoding and prefill therefore cost the same for a 4 s clip as for a 14 s clip on the 15 s file (or a 29 s clip on the 30 s file); only decoding grows with the transcript. With the 15 s file a clip longer than 15 s goes through two windows: the 28 FLEURS clips of 15.1–29.3 s in the set below were scored that way. Clips longer than 30 s were not tested with either file.

The answer is language <name><asr_text><transcript>. Split it at <asr_text>; the scores below use the text after the tag. On the FLEURS clips below every Mandarin and Japanese clip was tagged correctly; one English clip (two on the GPU run) came back as language None followed by the English transcript.

Forced language. Add a text item language Chinese<asr_text> (or another language name) to the same message. The template places it at the start of the answer, as the qwen-asr package does for a forced language, and the model then writes the transcript only (11 of 11 Mandarin and Japanese clips tried).

Prompt. The template renders <|im_start|>user<|audio_start|> + audio + <|audio_end|><|im_end|><|im_start|>assistant\n, the prompt of litert-torch's Qwen3-ASR export. The checkpoint's own chat template adds an empty system turn and newlines. With that template this file scored Mandarin CER 6.65 % and Japanese CER 6.06 % on the set below, and 8 of 50 Mandarin clips came back tagged English (with a Chinese transcript). System messages are ignored by the template.

Correctness

FLEURS test, 50 clips per language, ground truth raw_transcription. Greedy decoding, one conversation per clip.

Run	English WER	Mandarin CER	Japanese CER
Original model, PyTorch fp32 (transformers 5.14.1), this file's prompt	4.35 % (50/1150)	6.04 % (118/1954)	5.24 % (141/2689)
Original model, PyTorch fp32, the checkpoint's chat template	4.35 % (50/1150)	6.86 % (134/1954)	5.50 % (148/2689)
15 s file, LiteRT-LM 0.18.0 (Python), Apple M4 Max, CPU	4.35 % (50/1150)	6.19 % (121/1954)	5.84 % (157/2689)
15 s file, LiteRT-LM 0.18.0 (Python), Apple M4 Max, LM on the GPU (WebGPU over Metal), audio on the CPU	4.44 % (51/1150)	6.24 % (122/1954)	5.21 % (140/2689)
30 s file, LiteRT-LM 0.17.1 (Python), Apple M4 Max, CPU	4.43 % (51/1150)	6.19 % (121/1954)	5.73 % (154/2689)
30 s file, LiteRT-LM 0.17.1 (Python), Apple M4 Max, LM on the GPU (WebGPU over Metal), audio on the CPU	4.52 % (52/1150)	6.19 % (121/1954)	5.32 % (143/2689)

No clip came back empty. WER counts words after uppercasing and removing punctuation. CER counts characters after NFKC normalization and lowercasing, with all punctuation, separators and whitespace removed. The 30 s file on LiteRT-LM 0.18.0 scored 4.43 % / 6.40 % / 5.99 % on the CPU and 4.52 % / 6.19 % / 5.32 % on the GPU (the CPU transcripts of 20 of the 155 clips differ between the two runtime versions).

Performance

Response time per clip (both files)

One clip, warm process (the model loaded and the runtime caches present), greedy, the answer complete: the seconds cover the audio encoder, the prefill and the decoding of the transcript. The clip is the 6.4 s English FLEURS sentence 1698 shown above (23 output tokens); the Mandarin (9.7 s, 25 tokens) and Japanese (11.0 s, 38 tokens) clips of the same sentence are in parentheses. Mac: median of 5 calls in one process, LiteRT-LM 0.18.0 (Python), the machine's measurement window held but other processes running. Phones: the LiteRT-LM v0.18.0 prebuilt command-line tool, one process per clip, the phone rested to an unthrottled state before each process; on the phones the audio encoder runs on the CPU and its share moves with the phone's thermal state (on the S26, 0.25–0.76 s for the same clip and bytes within one session), so the phone rows are ranges over the processes measured. On the phones the Mandarin clip's process ran first in each session and wrote the runtime caches.

Device	Backend	15 s file	30 s file
Apple M4 Max	LM on the GPU (WebGPU over Metal), audio encoder CPU	0.34 s (0.36 / 0.46)	0.54 s (0.56 / 0.65)
Apple M4 Max	CPU, 4 threads	0.85 s (0.89 / 1.14)	1.19 s (1.24 / 1.50)
Galaxy S26 (SM-S942Q)	LM on the GPU (OpenCL), audio encoder CPU	1.40–1.51 s (1.60 / 2.07)	2.11–2.81 s (2.82 / 2.77)
Galaxy S26 (SM-S942Q)	CPU, 4 threads	1.55 s (2.18 / 2.17)	2.11 s (2.18 / 2.70)
Pixel 8a	LM on the GPU (OpenCL), audio encoder CPU	5.03–6.33 s (7.03 / 6.49)	9.86–9.98 s (11.89 / 10.01)
Pixel 8a	CPU, 4 threads	5.10 s (6.85 / 6.21)	8.39 s (11.95 / 9.48)

Galaxy S26, the 15 s file, LiteRT-LM v0.18.0 command-line tool with --benchmark, one process per clip, the prompt 203 tokens (195 audio + 8 text), the weight caches present:

Backend	Prefill	Decode	Time to first token	Peak private footprint
LM on the GPU (OpenCL, prefill_256 and decode fully delegated), audio encoder CPU	762–787 tokens/s	23.5–26.0 tokens/s	0.30–0.31 s	2.41 GB
CPU, 4 threads	511–571 tokens/s (one process 243)	22.9–25.7 tokens/s	0.39–0.44 s (one process 0.88 s)	4.10 GB

Time to first token covers the prefill and the first decode step; the audio encoder's time comes before it (0.25–0.29 s on these runs). On the GPU the whole 203-token prompt goes through the prefill_256 signature in one call; on the CPU the runtime feeds it through prefill_128 in two calls, so the CPU numbers of the 15 s file equal the 30 s file's per token.

The 30 s file on the Mac, 155 clips

Apple M4 Max, LiteRT-LM 0.17.1 (Python), the 150 FLEURS clips above plus 5 example clips (155 clips, 4–29 s, 1,836 s of audio). Other jobs were running on the machine (load average 4.5–8.5), so read the times as an upper bound:

Backend	Processing time	Real-time factor	Per clip	Peak RSS
CPU, 4 threads	272 s	0.148 (6.7× faster than real time)	1.04–3.17 s, median 1.70 s	4.54 GB ¹
LM on the GPU (WebGPU over Metal), audio encoder on the CPU	105 s	0.057 (17.5× faster than real time)	0.53–1.06 s, median 0.66 s	4.79 GB ²

¹ With the CPU weight caches already written. Peak RSS includes the memory-mapped weights. The first load writes about 3.0 GB of CPU weight caches (encoder 1.27 GB, LM 1.73 GB) into cache_dir, or next to the model file when none is set; a first load that wrote them peaked at 6.62 GB.

² This run was the first GPU load and wrote the GPU caches (1.72 GB) during the run.

The 30 s file on the Galaxy S26

Galaxy S26 (SM-S942Q), CPU, 4 threads, LiteRT-LM command-line tool built from the v0.16.1 release tag with --benchmark, one process per clip: an English, a Mandarin and a Japanese FLEURS clip, the English one twice. The prompt is 398 tokens (390 audio + 8 text). The CPU frequency limit was at its maximum before every run:

Run	Prefill	Decode	Time to first token	Peak private footprint
First run, writes the weight caches	329 tokens/s	22.8 tokens/s	1.25 s	3.47 GB
Next 3 runs, caches loaded	538–547 tokens/s	25.0–25.6 tokens/s	0.77–0.78 s	4.23 GB

All four transcripts equal the Mac CPU run.

Usage

Python (pip install litert-lm-api; the 30 s file was run on 0.17.1, the 15 s file on 0.18.0). transcribe takes a 16 kHz mono WAV of up to 30 s; prefix is text the answer starts with, for example language Chinese<asr_text> to force the language, and the return value is the answer after it. Open the file you downloaded; the code is the same for both:

import litert_lm
from litert_lm import Content, Contents, Message
from litert_lm.interfaces import CPU

engine = litert_lm.Engine("Qwen3-ASR-1.7B-15s.litertlm", backend=CPU(), audio_backend=CPU())
sampler = litert_lm.SamplerConfig(top_k=1, top_p=1.0, temperature=0.0)


def response_text(resp):
    # litert-lm-api 0.15-0.17.1 return a dict of content parts; later builds return a Message whose str() is its text
    if type(resp) is dict:
        return "".join(p.get("text", "") for p in resp.get("content", []) if isinstance(p, dict))
    return str(resp)


def transcribe(wav_path, prefix=""):
    items = [Content.AudioFile(wav_path)]
    if prefix:
        items.append(Content.Text(prefix))
    conv = engine.create_conversation(sampler_config=sampler)
    try:
        return response_text(conv.send_message(Message.user(Contents.of(*items))))
    finally:
        conv.close()


raw = transcribe("clip.wav")
print(raw.partition("<asr_text>")[2].strip())

For the Chinese example clip the 30 s file returns language Chinese<asr_text>开放时间：早上九点至下午五点。 and the script prints the text after the tag; the 15 s file returns the same sentence in traditional characters, 開放時間：早上九點至下午五點。 (both files run with this block on litert-lm-api 0.18.0; on the 50 Mandarin FLEURS clips, whose ground truth is simplified, the 15 s file scores CER 6.19 %). backend=GPU() (from litert_lm.interfaces) puts the language model on the GPU. Keep audio_backend=CPU(): the audio encoder does not run on the GPU delegate (create_conversation fails). The Kotlin Conversation API takes the same calls. In the Galaxy S26 app above (litertlm-android), the engine is built with Backend.CPU for both the model and the audio, and each clip is sent as Content.AudioFile in a new conversation.

Sample app (Android): android/ — one app for both files and two sibling bundles (Qwen3-ASR-1.7B, Qwen3-ASR-1.7B-15s, Fun-ASR-Nano-2512, Confucius4-R2T2): a model picker, hold-to-talk up to 30 s and three FLEURS clips, with a device check. Galaxy S26, language model and audio encoder on the CPU (4 threads), litertlm-android 0.17.1, wall time per call: the 30 s file 2.49 s for the 6.4 s English clip and 2.38 s for the 9.7 s Mandarin clip (2026-10-09); the 15 s file 1.79 s and 2.32 s (2026-10-14, where its device check passed all 14 steps: the three clips, a microphone recording, a 30 s message in two windows, release and reload). The sample's README has the full rows.

On Android the S26 rows above came from the command-line tool: litert_lm_advanced_main --backend=cpu --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --model_path=Qwen3-ASR-1.7B-15s.litertlm --cache_dir=cache --max_num_tokens=1024 --max_output_tokens=256 --benchmark --input_prompt='[audio:/path/clip.wav]' (--backend=gpu for the GPU rows; the 30 s file with its own file name).

License

Apache 2.0, the license of the original model.

Conversion

The conversion differs from a default export in three places:

1. Encoder attention windows. The original encoder attends within 104-token (8 s) windows. The default litert-torch export for Qwen3-ASR attends within 13-token (1 s) chunks. These files use the original windows through a constant block-diagonal attention mask; the exported encoder then matches the original encoder's output exactly in fp32 (maximum difference 0.0 on 5 s and 30 s inputs; with 13-token chunks the cosine similarity of the encoder output is 0.76–0.81).
2. Log-mel inside the encoder. The runtime's built-in mel front end uses a different filterbank and log scale than the original feature extractor, so the files ask the runtime for raw PCM frames and compute the original log-mel in the graph. On 10 FLEURS clips the in-graph mel of the 30 s file differs from the original by at most 0.028 (mean 0.00006, on features in about −1.5 to 1.6), which comes from the fp16 weights of the DFT and filterbank; the fp32 graph differs by at most 0.00003.
3. Quantization. int8 dynamic LM weights, an int8 embedding table and fp16 encoder weights: the recipe chosen for Confucius4-R2T2, a fine-tune of this model with the same audio encoder, and checked here on the 155 clips above (every language within 0.6 points of the original model in PyTorch with the same prompt).

The 15 s file (2026-10-13) came out of a speed ladder run on the 30 s file: the audio encoder re-exported with a 15 s window (input [1, 1500, 160], output [1, 195, 2048]; same weights, same fp16 recipe) and the language model re-exported with prefill signatures for 128 and 256 tokens (--prefill_lengths 128,256; same int8 recipe). Of the other forms tried, int4 (blockwise 32) LM weights failed the correctness check (the model stops after the language tag on half of the 155 clips), a dynamic-int8 encoder with the log-mel kept in float passed it with no gain on the GPU, and a 512-token KV cache passed on the CPU but failed on the GPU for Japanese.

These files are made for the Conversation API. The omni/asr runner in the LiteRT-LM repository was not run with them.

Scripts and a one-command reproduction: https://github.com/john-rocky/hf-to-litertlm/tree/main/qwen3_asr17_work.