Maya1-ONNX-WebGPU

Built with Llama.

Maya Research's Maya1 text-to-speech model, with the decoder of the SNAC 24 kHz audio codec, as ONNX graphs with quantized weights for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference. A Llama 3.2 3B-based model writes SNAC codes (7 tokens per 85 ms frame) in a voice described in words; the decoder turns them into 24 kHz audio. English, with inline tags such as <laugh>, <sigh>, <whisper>, <gasp> and <sing> (single tokens; the list is in the tokenizer's added tokens after the audio tokens).

Changes from the original

Converted in Quick Inference:

• The language model: ONNX Runtime GenAI's model builder, -p int8 -e webgpu, block size 32: 8-bit weights (round to nearest) in every linear layer including the output layer; float16 embeddings, activations and KV cache. It is 8-bit rather than 4-bit because every 4-bit build made the model speak about 40% faster than the original.
• The SNAC decoder: from onnx-community/snac_24khz-ONNX's float decoder_model.onnx, convolution weights quantized to 4 bits without calibration (round to nearest, symmetric, blocks of 32).

These are modified files: their outputs are close to, but not identical to, the original model's. On four test sentences (the description below, the sampling below), Whisper small.en heard them with a 7.1% word error rate (ONNX Runtime CPU, and the same in Chrome), as for the original model; the clips are about 10% shorter.

Files

• onnx/model.onnx, onnx/model.onnx.data, onnx/genai_config.json: the language model. input_ids int64 [1, S], attention_mask int64 [1, P + S], past_key_values.{0..27}.{key,value} float16 [1, 8, P, 128] → logits float16 [1, S, 156960], present.*.
• onnx/snac_decoder_q4.onnx: audio_codes.0/1/2 int64 [1, n], [1, 2n], [1, 4n] → audio_values float32 [1, 1, 2048 n].
• tokenizer.json, tokenizer_config.json: the original's.

Generation (the model card's)

1. ids = [128000, 128259, 128000] + tokens('<description="{description}"> {text}') + [128009, 128260, 128261, 128257], for example with the description Realistic female voice in the 30s age with an american accent. Normal pitch, warm timbre, conversational pacing, neutral tone at medium intensity.
2. Sample (repetition penalty 1.1 over the prompt and the tokens so far, temperature 0.4, top-p 0.9) until 128258, which is masked for the first 28 tokens.
3. Audio tokens are 128266 + (i % 7) × 4096 + code; every 7 form a frame [L1, L2a, L3a, L3b, L2b, L3c, L3d].
4. Decode all frames with SNAC and drop the first 2048 samples.

Use

Maya Research asks users to disclose synthetic voice where appropriate, and not to use generated audio for deceptive impersonation, fraud, fabricated evidence or unauthorized representations of a real person's statements.

Licence

• Maya1: Apache 2.0 (LICENSE). It is built on Llama 3.2 3B, so the Llama 3.2 Community License applies too (LICENSE-Llama-3.2.txt), with Meta's Acceptable Use Policy. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
• SNAC: MIT (LICENSE-SNAC).