maya2-native-fast

Optimized CPU inference for maya-research/maya2-native: ONNX graphs, int8 backbone, fp16 depth, a torch-free numpy runtime.

This is not a fine-tune. The weights are converted from the upstream checkpoint. No training data was used.

What is in the repo

Path	Contents
assets/v1/backbone_wo8.onnx + .data	28-layer Qwen3 backbone, int8 weights (MatMulNBits, block 32), 1.6 GB
assets/v1/depth_frame_fp16.onnx + .data	Depth sampler, 15 levels, fp16 storage, Gumbel top-k sampling in-graph, 873 MB
assets/v1/mimi_decoder.onnx	Mimi decoder, fp32, dynamic frame count, 195 MB
assets/v1/embed_bf16.bin, audio_in_bf16.bin	Text and audio embedding tables, bf16, memory-mapped
assets/v1/prompt.npz, tokenizer.json, voices.json, controls.json	Prompt weights, extended tokenizer, voices, sentence controls
assets/v1/manifest.json	sha256 of every file above, runtime shapes. The runtime refuses mismatches.
src/maya_fast/	Runtime (numpy, onnxruntime, tokenizers; no torch)
tools/maya_export/	Export and evaluation scripts, to rebuild the assets from the upstream checkpoint
tests/	Runtime and export tests

Measured results (Apple M-series Mac, CPU, 8 threads)

Check	Result	Target	Status
Depth fp16 vs fp32 reference, teacher-forced, 75 files	top-1 >= 0.9965 every level, KL <= 6e-6	top-1 >= 0.98, KL <= 0.02	pass
Depth fp32 vs upstream PyTorch, 20 frames	rel 1.6e-6	rel <= 1e-4	pass
Mimi decoder vs upstream	max abs 1.5e-6	<= 1e-4	pass
Prompt port vs upstream, 500 prompts	0 token mismatches, max embed error 7.5e-9	<= 1e-5	pass
Backbone int8 vs fp32, teacher-forced	top-1 0.9677, KL 0.0034	top-1 >= 0.99	fail (waived)
Backbone speed, 8 threads	48 / 58 / 82 ms per token at 32 / 256 / 768 cached tokens	<= 35 ms at 256	fail (waived)
End-to-end, one Marathi sentence (2.96 s audio), two runs	RTF 4.67 and 4.43	offline RTF <= 4	fail
Peak RSS, end-to-end	3.9 GB	<= 3 GB target, 5 GB limit	over target

Known limits:

• The int8 backbone misses top-1 agreement; expect small differences in generated codes.
• End-to-end RTF on this machine is above the offline target.
• Hindi (--lang hi) works mechanically; it has not been checked for quality.
• Not yet checked by listening tests.
• Real-time playback is not supported.

Speed

Measured on an Apple M-series Mac, CPU only, 8 threads. All numbers are from this repo's runs.

Stage	Measurement	Notes
Backbone (int8, 28 layers)	48 ms per token at 32 cached tokens, 58 ms at 256, 82 ms at 768	median of 200 steps, one token per step
Depth (fp16 sampler, 15 levels)	155 ms per frame	median of 30 calls, one frame
Mimi decoder	about 1 ms per frame	80 ms of audio per frame
End-to-end, one Marathi sentence (2.96 s audio)	RTF 4.67 and 4.43 (two runs), 13.1 to 13.8 s wall	includes prompt, prefill and decode

For comparison, the upstream PyTorch path on the same kind of CPU measured in this project's spike: RTF 4.62 (best, bf16) and peak RSS 7.5 to 8.0 GB.

What this means:

• Speed is about the same as upstream PyTorch on CPU. Neither is real-time (RTF is above 1).
• Peak RSS is about half the upstream figure (3.9 GB here).
• The gain is memory and dependencies (no torch at runtime), not throughput.
• Real-time playback is not supported.

Install

python -m venv .venv
.venv/bin/pip install -r requirements-fast.txt

Requires Python 3.10 to 3.13. Peak memory is about 4 GB while a sentence is synthesized.

Use: command line

Run from the repo root (so src is importable):

PYTHONPATH=. .venv/bin/python -m src.maya_fast "नमस्कार, मी तुमची मदत करण्यासाठी इथे आहे." \
    --assets assets/v1 -o out.wav

Options:

• --voice NAME: named voice from assets/v1/voices.json (case-insensitive). Default Charvi.
• --lang mr|hi: language tag. Default mr.
• --seed N: random seed. Same seed and threads give the same audio.
• --threads N: ORT threads. Default min(8, cores).

Use: Python

from src.maya_fast.engine import MayaFast

engine = MayaFast(assets_dir="assets/v1", threads=8)
wav, metrics = engine.synthesize("नमस्कार, मी तुमची मदत करण्यासाठी इथे आहे.", voice="Charvi", seed=0, lang="mr")
# wav: float32, 24 kHz mono

Write it with soundfile or the standard wave module.

Assets are verified against manifest.json at load. A bad or partial download fails with the file name.

Rebuild the assets from upstream (optional)

Needs the upstream checkpoint (maya-research/maya2-native, kyutai/mimi, speechbrain/spkrec-ecapa-voxceleb) and an upstream maya3 install in a second venv (.venv-tts, torch 2.8, transformers 5.6.2). The export scripts write to $MAYA_FAST_WORK/assets/v1, default ~/.cache/maya_fast. See tools/maya_export/:

1. export_prompt_assets.py, export_backbone.py --precision wo8, export_depth_all.py, export_codec.py
2. finalize_assets.py writes the runtime manifest. Run each step under tools/memguard.py, which kills the process above a memory limit.

License and attribution

• These weights and code are derived from Maya2 Native (Apache-2.0) by Maya Research.
• The Mimi decoder is derived from Kyutai Mimi (CC-BY-4.0). Keep this attribution.
• The text backbone is derived from Qwen3-1.7B-Base (Apache-2.0).
• Speaker conditioning uses SpeechBrain ECAPA-TDNN (Apache-2.0).
• Full notices in NOTICE.

The model adds no audio watermark.

Citation

Please cite the upstream model:

@misc{maya2native2026,
  title  = {Maya 2 Native: Streaming Multilingual Text-to-Speech},
  author = {{Maya Research}},
  year   = {2026},
  url    = {https://huggingface.co/maya-research/maya2-native},
  note   = {Release 1.0}
}