cowWhySo/MegaTTS3-TurboSpark

MegaTTS3 for TurboSpark (MLX-layout safetensors)

ByteDance's MegaTTS3 zero-shot voice cloning model as MLX-layout safetensors, converted for TurboSpark, which runs it natively on Apple Silicon (on Metal by default). Give it 1 to 30 seconds of someone's voice and some English text, and it speaks the text in that voice at 24 kHz. The weights are the upstream values in a new file format, not a retrain or a fine-tune.

MLX layout means the converted tensors follow MLX's conventions: f32, with conv weights stored as [out, kernel, in]. It is not an MLX model: mlx-audio has no MegaTTS3 port, so these files will not load there, and the upstream PyTorch code cannot load them without reversing the conversion. TurboSpark's Rust runtime loads them directly. If you are not using TurboSpark, you probably want the upstream release instead.

Why this exists

TurboSpark is a hobby project: a native macOS app and Rust workspace for running local models on Apple Silicon. Its runtime reads safetensors only, and its model catalog installs only files pinned to a commit and a sha256. MegaTTS3 ships as PyTorch pickle checkpoints, which TurboSpark will not load, because opening a pickle can run code.

Converting once and publishing the result lets the app install MegaTTS3 like any other model and check every byte before it runs. It also means you can rerun the converter and compare hashes rather than trusting this upload.

What is in the repo

Path	What it is	Params	Size
config.json	Root config. TurboSpark dispatches on model_type: megatts3, and voice_clone says whether the WavVAE encoder is present.		133 B
aligner_lm/	Whisper-style aligner. Finds the phones and their timing in the reference recording.	54.6M	218 MB
duration_lm/	Duration model. Predicts how long each phone of the new text lasts.	67.0M	268 MB
diffusion_transformer/	24-layer latent diffusion transformer. Generates the speech latents in the reference voice.	459.0M	1.84 GB
wavvae/	WavVAE codec. The encoder turns the reference into a latent, and the decoder turns latents into 24 kHz audio.	255.6M	1.02 GB
g2p/	Qwen2-0.5B grapheme-to-phoneme model and tokenizer, copied unchanged (bf16).		1.04 GB
frontend/dict.json	Upstream phone and tone dictionary. Kept for reference. TurboSpark does not read it.		1.75 KB
manifest.json	sha256, size, and tensor count of every file, plus the upstream sources.
NOTICE.md	Exact upstream sources, their sha256, and where the encoder came from.		1.63 KB
LICENSE	Apache-2.0, from the upstream MegaTTS3 code.		11.3 KB

Each component folder also has a config.json with the upstream hyperparameters and the sha256 of the checkpoint it came from.

How it was converted

Every file here comes from crates/audio/tools/convert_megatts3.py in the TurboSpark repo.

• It checks each upstream file against a pinned sha256 before reading it.
• It parses the checkpoints with an allowlisted unpickler and never imports torch.
• It widens f16 and bf16 tensors to f32, which loses nothing.
• It stores conv weights as [out, kernel, in], the layout MLX uses, and keeps weight norm as separate weight_g and weight_v tensors.
• It writes the same bytes for the same inputs, so manifest.json doubles as a reproducibility check.

Sources, also listed with their hashes in NOTICE.md:

• ByteDance/MegaTTS3 at 409a7002: the aligner, duration model, diffusion transformer, and G2P.
• mrfakename/MegaTTS3-VoiceCloning at b3a2c297: the WavVAE with its encoder.

Using it with TurboSpark

Voice cloning is on TurboSpark's main branch and not in a release yet. Every entry point asks you to confirm that you have the right to use the voice.

• App. In Audio Studio, open Voice Clone, pick MegaTTS3, accept the download notice, choose or record a reference, and pick Fast, Balanced or Best.
• Command line. turbospark-clone takes this folder or the installed alias. It runs on Metal on macOS, --quality fast|balanced|best picks 8, 16 or 32 steps, and --cpu runs the slower CPU reference path:

hf download cowWhySo/MegaTTS3-TurboSpark --local-dir MegaTTS3-TurboSpark
turbospark-clone speak --model MegaTTS3-TurboSpark --reference me.wav \
  --text "Hello there." --consent --output hello.wav

• HTTP. turbospark-server --tts-model serves it on the OpenAI-style speech route, with the reference as a base64 WAV:

turbospark-server --tts-model MegaTTS3-TurboSpark
curl http://127.0.0.1:8080/v1/audio/speech -H 'content-type: application/json' \
  -d "{\"model\":\"MegaTTS3-TurboSpark\",\"input\":\"Hello there.\",\"consent\":true,\"reference_audio\":\"$(base64 -i me.wav)\"}" \
  -o hello.wav

Optional settings are steps (default 16, with 8 as Fast and 32 as Best), p_w (intelligibility, default 1.4), t_w (similarity to the reference, default 3.0), seed, and speed.

Benchmarks

Measured with TurboSpark on one Apple M4 Max (36 GB). At the default settings the Metal path takes about 5 s per second of speech and peaks near 9 GiB for a short request. The timing ran on battery with other work on the machine, so read it as ratios between rows, not absolute numbers. Method and raw results are in the port's README.

Speed

One 3.04 s sentence cloned from a 2.8 s reference. Each row is the median of three interleaved runs, each a fresh process after a discarded warmup. RTF is seconds of compute per second of speech.

Device	Steps	Time	RTF
Metal	8 (Fast)	12.7 s	4.2
Metal	16 (Balanced, default)	15.5 s	5.1
Metal	32 (Best)	24.6 s	8.1
CPU	32	128.5 s	42.3

Memory

Peak process footprint at 32 steps, the figure macOS charges the app, one run per request:

Request	Metal	CPU
2.8 s reference, 3.0 s of speech	8.6 GiB	6.6 GiB
7.0 s reference, 7.4 s of speech	9.8 GiB	8.4 GiB
27.5 s reference, 13.3 s of speech	14.3 GiB	11.4 GiB

Metal holds 1.4 to 2.9 GiB more because the uploaded weights keep a CPU copy. TurboSpark checks memory before every request and refuses one that would not fit. By that check, a 16 GB Mac would run the short request and refuse the 7 s one on either device. That is arithmetic from these numbers, not a run on 16 GB hardware.

Accuracy

• Against PyTorch. With every random draw replayed, TurboSpark's CPU path matches the upstream PyTorch pipeline on these weights: max sample difference 5.4e-5 and 1.1e-3 on two cases, correlation 1.000000.
• Metal against CPU. Same seed, 32 steps, 8 LibriSpeech test-clean speakers and two multi-chunk passages: identical lengths, RMS difference at most 2.3e-5, and a largest single-sample difference of 1.8e-3 (on a 13 s passage).
• Steps. 8 LibriSpeech speakers, two sentences each, on Metal. WER is distil-whisper large-v3. Similarity is the cosine of WavLM-base-plus-sv embeddings against the reference: a second real recording of the same speaker scores 0.963, and two different speakers 0.774.

Steps	WER	Similarity
4	8.5%	0.968
8 (Fast)	6.4%	0.971
16 (Balanced)	6.8%	0.976
32 (Best)	6.4%	0.974
32, another seed	8.2%	0.973

From 8 steps up, no difference is larger than changing the seed. Most of the WER is rare names and archaic words in the LibriSpeech texts, which every setting misreads alike. None of this is a listening test.

Consent and the encoder

ByteDance withheld the WavVAE encoder, the part that turns a recording into a voice the model can imitate, over misuse concerns. The encoder here is a third-party re-upload by mrfakename, credited to ModelScope ACoderPassBy/MegaTTS-SFT. Its origin and license were not independently verified, and the re-upload's repository declares Apache-2.0.

TurboSpark asks you to accept that before it downloads these files, and asks on every request whether you have the right to use the voice. Please only clone your own voice or the voice of someone who agreed to it.

Limits

• English only. TurboSpark's text normalizer covers a documented subset of upstream's, and non-Latin text is refused.
• The reference can be 1 to 30 seconds, and only the first 28 are used. Shorter references synthesize faster.
• Output is mono 24 kHz.
• It is not real time: about 5 s of compute per second of speech at the default settings on Metal (see Benchmarks).
• No listening test or MOS has been run.

The measurements behind these numbers are in the port's README.

Credits

MegaTTS3 is ByteDance's work: paper, code, and weights. Thanks to them for releasing it under Apache-2.0, and to mrfakename for the encoder re-upload.