acul3/omnivoice-executorch-qnn-sm8750

OmniVoice — ExecuTorch / Qualcomm NPU build (Snapdragon SM8750)

On-device text-to-speech and zero-shot voice cloning for k2-fsa/OmniVoice, converted to ExecuTorch and running on the Qualcomm Hexagon NPU (HTP V79) of the Snapdragon 8 Elite (SM8750), plus a ready-to-install Android app.

OmniVoice is a single-stage masked-diffusion TTS model: a Qwen3-0.6B bidirectional transformer predicts 8 codebooks of Higgs-audio tokens over 32 iterative steps, which a Higgs vocoder turns into 24 kHz audio. This repo splits it into three ExecuTorch programs and runs the heavy predictor on the NPU.

Built with Claude Code. Models derive from the Apache-2.0 base model above.

Requirements: these .pte are offline-compiled for HTP V79 / Snapdragon 8 Elite (SM8750) and will not load on other SoCs without recompiling (see below). You also need the QNN aarch64 runtime libs on the device (from the QAIRT SDK) and an ExecuTorch QNN runner. The full run/build flow is in "How to run".

---

Files

Use these (recommended pipeline):

File	Role	Backend	Notes
omnivoice_predictor_qnn_fp16_s256.pte	Diffusion predictor, seq-len 256	QNN FP16 / HTP V79	Fast default; short prompts (~5 s/32 steps)
omnivoice_predictor_qnn_fp16.pte	Diffusion predictor, seq-len 512	QNN FP16 / HTP V79	Long text / voice clone (~19 s/32 steps)
omnivoice_audio_decoder_xnnpack_dyn.pte	Token → 24 kHz waveform	XNNPACK CPU (dynamic length)	Sub-second; handles any output length
omnivoice_audio_encoder_xnnpack_5s.pte	Reference audio → tokens (clone)	XNNPACK CPU	Fixed 5 s input (120000 samples @ 24 kHz)
tokenizer.json	Qwen3 text tokenizer (+OmniVoice control tokens)	—	Bit-exact vs HF transformers

Do NOT use (kept for transparency):

File	Why not
omnivoice_audio_decoder_qnn_fp16.pte	The FP16 vocoder is numerically unstable on HTP (correct sample count, garbage audio). Use the XNNPACK decoder.
omnivoice_predictor_qnn_16a8w.pte	INT8 (16a8w) is too lossy for this diffusion model (≈0.5 % token agreement vs FP16, garbage audio). FP16 is the floor.
omnivoice_audio_decoder_cpu.pte	Legacy fixed-75-token portable decoder (~60 s). Superseded by the dynamic XNNPACK one.

---

How it works

text ──HF tokenizer──► conditioning ─┐
ref audio ──encoder(CPU)──► ref tokens ┘
                                       ▼
        32× iterative masked diffusion (predictor on NPU, classifier-free guidance)
                                       ▼
                       8×T audio tokens ──decoder(CPU)──► 24 kHz WAV

• Sampling: 32 steps, time-shifted schedule (t_shift=0.1), classifier-free guidance scale=2, Gumbel position temperature 5, layer penalty 5, greedy token assignment. (These are OmniVoice defaults — required for quality; greedy-without-Gumbel or fewer steps degrade output.)
• Audio: 24 kHz, 8 codebooks, ~25 tokens/s. Output length is auto-estimated from the text (capped to fit the predictor's sequence length).
• Why split across backends: the predictor is the heavy, repeated compute → NPU (FP16). The vocoder is FP16-unstable on HTP → CPU FP32 (XNNPACK, still sub-second). The encoder's conv stack hits a QNN pass limitation → CPU.

---

How to run (on-device)

The C++ runner, JNI library, an example Android app, and all export/build scripts are bundled here as omnivoice_executorch_src.tar.gz (also upstreamable into ExecuTorch under examples/qualcomm/oss_scripts/omnivoice/).

Build the runner (host = Linux x86_64):

1. Install QAIRT/QNN SDK 2.37+ and Android NDK r26c.
2. Clone ExecuTorch, run ./install_executorch.sh, then build the QNN backend: backends/qualcomm/scripts/build.sh --release (auto-fetches the SDK/NDK if unset).
3. Copy the bundled omnivoice/ dir into examples/qualcomm/oss_scripts/, register it in examples/qualcomm/CMakeLists.txt (add_subdirectory(... oss_scripts/omnivoice)), and build: cmake --build build-android/examples/qualcomm --target omnivoice_runner.

Push to the device /data/local/tmp/omnivoice/: the omnivoice_runner binary, these .pte files, tokenizer.json, libqnn_executorch_backend.so, and the QNN aarch64 libs (libQnnHtp.so, libQnnHtpV79Stub.so, libQnnSystem.so, hexagon-v79/unsigned/libQnnHtpV79Skel.so, libc++_shared.so). libcdsprpc.so must be reachable (it is in /vendor/lib64 on Snapdragon devices). Then:

# 1) prepare conditioning on a host (exact tokenizer) -> omnivoice_inputs.bin
python -m examples.qualcomm.oss_scripts.omnivoice.prepare_inputs \
  --text "Hello from OmniVoice." --language en --target_tokens 75 --seq_len 256 \
  --out omnivoice_inputs.bin

# 2) run on device (predictor on NPU + CPU decoder) -> WAV
LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH=. ./omnivoice_runner \
  --predictor omnivoice_predictor_qnn_fp16_s256.pte \
  --decoder  omnivoice_audio_decoder_cpu.pte \
  --inputs omnivoice_inputs.bin --output out.wav --seq_len 256 --num_step 32

(The Android app does the tokenizer + bucket selection + XNNPACK decode for you; the standalone CLI uses a host-prepared input blob and the portable decoder.)

---

Reproducing the .pte from the base model

# predictor (FP16, padded seq-len 512 — repeat with --seq_len 256 for the fast bucket)
python -m examples.qualcomm.oss_scripts.omnivoice.omnivoice \
  --components predictor --soc_model SM8750 --compile_only \
  --ptq none --seq_len 512 --model_dtype fp32

# decoder (dynamic length, XNNPACK) and encoder (fixed 5 s, XNNPACK)
python -m examples.qualcomm.oss_scripts.omnivoice.export_decoder_dynamic --backends xnnpack
python -m examples.qualcomm.oss_scripts.omnivoice.export_encoder_dynamic  --backends xnnpack --calib_seconds 5

Recompiling for another Snapdragon SoC

These context binaries are compiled for HTP V79 (SM8750). For another SoC, re-run the predictor/decoder/encoder export with --soc_model <YOUR_SOC> (see QcomChipset in the ExecuTorch QNN backend) on a Linux x86_64 host with QAIRT.

---

Limitations

• SM8750 / HTP V79 only (offline-compiled). Other SoCs require recompiling.
• App v1 is English; the base model supports 600+ languages (expose via the language/instruct tokens).
• Voice-clone reference is a fixed 5 s window (padded/truncated).
• FP16 only — INT8/INT4 degrade this diffusion model too much; FP8 is not available on HTP.

Credits

• Base model: OmniVoice by Xiaomi / k2-fsa (Apache-2.0).
• Audio tokenizer: Higgs-audio v2.
• On-device port: ExecuTorch + Qualcomm QNN. Built with Claude Code.