Audio8-TTS-Realistic-Air-MLX-Codec-Q8

This is a third-party MLX conversion of Edge0/Audio8-TTS-Realistic-Air for local text-to-speech and reference-conditioned voice cloning on Apple silicon. The main language and acoustic-token generator retains its original BF16 tensor values. Quantization applies to the separate 44.1 kHz audio codec. Pinned upstream revision

• model.safetensors stores the upstream model.* BF16 tensors.
• codec.safetensors stores affine Q8 weights with a group size of 64 for eligible linear, Conv1d, and transposed Conv1d layers. The remaining codec tensors use FP16. The file contains 359,230,364 bytes with SHA-256 67e7e4e5dad0893a2ad7b47c531aa28a9b6ae1bed64ab581a4f764bc2c09ec64.
• infer.py loads this format using qconv.py and qtranspose.py. Use this runtime with the supplied configuration and tokenizer. The packed convolution layout requires these operators.

Run locally

Download this repository and run the following from its directory on an Apple silicon Mac. The dependency file pins MLX 0.32.4 and the upstream MLX Audio revision used for this conversion.

python -m pip install -r requirements.txt
python infer.py --model-dir . --text '早安，今天一起聽故事。' --output speech.wav

To clone a voice with the speaker's consent, supply your own reference recording and its matching transcript.

python infer.py --model-dir . --text '早安，今天一起聽故事。' \
  --reference-audio reference.wav --reference-text-file transcript.txt \
  --output cloned.wav

The output is a mono 44.1 kHz WAV. Use --text-file for longer input.

Measurements

The Python MLX 0.32.4 prototype was measured on macOS Metal with a 30-second reference and a single Mandarin voice. The peak is the whole-process memory footprint reported by /usr/bin/time -l, in decimal GB. The long run used 1,000 input characters split into 25 segments, each at most 58 characters.

Workload	FP32 codec	Q8 codec
Short sentence peak	6.818 GB	2.771 GB
1,000-character peak	—	2.763 GB
1,000-character generated audio	—	242.09 s

The short sentence yielded 0.00% CER with the FP32 codec and 0.00% with Q8 using Whisper large-v3-turbo. ECAPA speaker cosine versus the reference was 0.757 with FP32 and 0.768 with Q8. Fixed-token decoding yielded 36.74 dB waveform SNR relative to the FP32 codec. Quantizing the reference encoder changed 2,126 of 6,460 codec indices in this test. The full voice-cloning output can therefore differ even with identical BF16 generator weights.

These measurements cover a single reference and Mandarin workload. The 1,000-character run measured completion and memory. Speech quality was scored on the short sentence. The upstream model card documents its language coverage. Mobile memory limits and performance require separate on-device measurements. Upstream model card

License and attribution

The model and derived codec retain the upstream Apache License, Version 2.0. Original model by Audio8 and Edge0. ShinePixelOrg converted the codec and repackaged the BF16 generator tensors. Bundled license · Bundled upstream notice · Original pinned license