phoonnx-magpie-tts

ONNX export of nvidia/magpie_tts_multilingual_357m (MagpieTTS Multilingual, v2607) for use with phoonnx.

The original model is a transformer encoder-decoder that predicts discrete audio codec tokens autoregressively. It uses 8 codebooks, a frame stacking factor of 2, and a local transformer that refines each stacked frame. A frozen NanoCodec decodes the tokens into a 22.05 kHz waveform.

Weights and architecture are NVIDIA's. This repository only changes the file format.

License and attribution

The model is released under the NVIDIA Open Model License. That license applies to every file here. Developed by NVIDIA. See the original model card for the full model card, dataset list, and limitations.

Contents

File	Purpose
text_encoder.onnx	6-layer causal text encoder. Text token ids to encoder states.
cross_kv.onnx	Per-layer cross-attention K/V, computed once from the encoder states.
decoder_step.onnx	12-layer causal decoder with external KV cache. Returns codebook logits, decoder states, the updated cache, and per-layer cross-attention probabilities.
local_step.onnx	One step of the 2-layer local transformer that refines the 16 stacked codebook tokens.
audio_embed.onnx	Stacked codebook tokens to one decoder input embedding.
lt_embed.onnx	One codebook token to the next local-transformer input.
codec_decoder.onnx	NanoCodec decoder. Codes to waveform.
assets/context_embeddings.npy	Baked speaker context embeddings, shape (5, 217, 768).
assets/tokenizer.json	Aggregated tokenizer vocabulary and per-language offsets.
assets/tokenizer/	IPA dictionaries and heteronym lists shipped inside the original checkpoint.
config.json	Graph inventory and IO contract.

All graphs are opset 18 and use external data files (*.onnx.data). Download the whole repository, not single files.

Two decode modes

NeMo ships use_kv_cache_for_inference: false. In that mode the newest attention prior is re-applied over the whole history, so the decoder states must be recomputed every step. A KV-cached decoder does not produce the same tokens.

decoder_step.onnx supports both:

• exact - pass the full sequence with an empty cache every step. Matches the NeMo default.
• cached - pass one frame plus the cache. Matches NeMo with use_kv_cache_for_inference: true. Roughly 2.5x faster.

Both reach 100% greedy token agreement with the matching NeMo reference across all 8 codebooks.

Usage

from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
voice = manager.get_voice(magpie/Leo/en)
wav = voice.synthesize(Hello from Magpie running on ONNX.)

Related

• phoonnx - the ONNX TTS runtime that consumes this repository
• nvidia/magpie_tts_multilingual_357m - the source checkpoint
• nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps - the codec