litert-community

/

wav2vec2-base-960h-LiteRT

Like 1

Follow

LiteRT Community (FKA TFLite) 9.11k

Hi @thuongvv — thanks for the precise list. The espeak checkpoint is a Wav2Vec2ForCTC like this one, so the recipe carries over: the build script is public, the GPU work is four architecture-level rewrites, fp16 is what runs on the GPU, and int8 is something I have not built for this model. Two things are not yet verified on a phone for a large-lv60 model; they are at the end.

Recipe. asr/scripts/build_w2v2_asr.py in my LiteRT-Models repo is the exact build behind the two files here (16 s window, frontend + head); build_w2v2_ctc.py next to it holds the rewrites. W2V2_MODEL_ID=facebook/wav2vec2-lv-60-espeak-cv-ft python build_w2v2_asr.py points it at your checkpoint; the CTC head width, layer count, feat_extract_norm and do_stable_layer_norm are read from the config. The same rewrites are in the official samples repo as the wav2vec2 KWS recipe.

Rewrites for the GPU (litert-torch export; CompiledModel on Accelerator.GPU has no CPU fallback, so one unsupported op fails the whole compile):

• GELU → tanh-GELU: exact GELU lowers to Erf, which the GPU rejects. The only non-exact step; the exported graph matches the re-authored model at corr 1.000000.
• Feature-extractor GroupNorm → 4-D reshape group-norm (the direct lowering emits GATHER_ND). Your checkpoint has feat_extract_norm: layer, so this one is unused.
• pos_conv weight-norm folded into a static weight; the runtime g·v/‖v‖ recompute splits the GPU partition.
• create_bidirectional_mask → None: the encoder builds an all-valid mask even with attention_mask=None (SELECT_V2 + BROADCAST_TO). With a fixed window and no padding it is a no-op, so attention becomes plain SDPA; short clips are zero-padded and decoded over the valid frames.
• Deployment: at 16 s the fused graph exceeds the Mali whole-graph shader-compile ceiling on a Pixel 8a although every op is supported, so it ships as two graphs split at the conv-frontend / encoder boundary (the 10 s single graph still compiles). A 24-layer, 1024-wide head is over three times this one; expect one more split.

fp16 / int8. fp16 is the shipped precision, and the Mali GPU computes in fp16 either way. The base model's residual peaks at |x|≈3.2, so it is fp16-exact: device logits match the desktop float run at corr 0.9928. I have not measured the large-lv60 activation range; that is the first thing to check on your model. int8 I have not built for wav2vec2, but two measured facts apply. Dynamic-range int8 fully-connected layers on a rank-3 input whose output feeds an ADD — the residual in every transformer block — fail the Mali GPU compile (LiteRT#9277, open; measured on a conformer ASR on the same Pixel 8a), so a DRQ-int8 wav2vec2 would be CPU-only there. And the Snapdragon NPU (Galaxy S26, Hexagon) runs the fp16 head as-is at 76 ms per 16 s window, so int8 is not needed to reach it. For full-integer int8 I have no calibration set to recommend from experience; the honest advice is a few hundred utterances from your deployment distribution (Common Voice for this checkpoint) at the fixed window, gated on per-frame argmax agreement against the float model.

Other checkpoints. Anything Wav2Vec2ForCTC converts the same way. Base-size (12-layer) checkpoints are device-verified; a large one is not yet — the compile ceiling above and the fp16 range are the two unknowns. If it helps, I can run the espeak checkpoint through the build and post the op-check, parity and Pixel 8a result here.

Links:

• Build behind this repo, with the rewrites: https://github.com/john-rocky/LiteRT-Models/blob/main/asr/scripts/build_w2v2_asr.py (build_w2v2_ctc.py next to it)
• The same rewrites in the official samples repo (KWS variant): https://github.com/google-ai-edge/litert-samples/blob/main/models/wav2vec2/wav2vec2_kws/converted/build_w2v2.py
• The DRQ-int8 GPU limitation: https://github.com/google-ai-edge/LiteRT/issues/9277

Thanks again for asking here — happy to look at whatever the espeak build turns up.

![IMG_20260917_010827_812](https://cdn-uploads.huggingface.co/production/uploads/6a5fc48b73cfd174c50cfec4/8iLmA8e8EVohkYibyn6co.
git clone https://huggingface.co/spaces/selfit-camera/Omni-Image-Editor
cd Omni-Image-Editor