remember2015/omnivoice-mlx-q8-fp16

OmniVoice, int8 + fp16, for MLX

k2-fsa/OmniVoice converted for omnivoice-mlx, a torch-free MLX port for Apple silicon. The Qwen3 backbone's Linear layers are quantised to int8 with group size 64, and activations are fp16. The eight codebook output heads are left unquantised in fp32: quantising them buys no speed and costs naturalness (UTMOS 2.835 → 2.794). 829 MB, 1.6 GB resident with the codec.

source	framework	device	weights	activations	steps	RTF	notes
official	torch	CPU	fp32	fp32	32	≈2–3
official	torch	MPS	fp32	fp32	32	0.98
mlx-audio	MLX	GPU	bf16	bf16	32	0.68
this	MLX	GPU	int8	fp16	32	0.381
this	MLX	GPU	int8	fp16	16	0.106	default
this	MLX	GPU	int8	fp16	16	0.080	batched B=4–8

CER, speaker similarity and UTMOS are unchanged from the official 32-step sampler. M2 Max 12-core CPU / 38-core GPU / 32 GB, macOS 26.6, MLX 0.32.2. Method, ablations and dead ends are in the port's research log (Chinese).

Use

pip install git+https://github.com/Remember2015/omnivoice-mlx
hf download remember2015/omnivoice-mlx-q8-fp16 --local-dir models/mlx-q8-fp16

The codec weights have their own licence and are not in this repository; take them from upstream:

hf download k2-fsa/OmniVoice --local-dir models/k2-fsa-OmniVoice
ln -s ../k2-fsa-OmniVoice/audio_tokenizer models/mlx-q8-fp16/audio_tokenizer
python -c "from omnivoice_mlx.codec import write_slim_decoder; write_slim_decoder('models/mlx-q8-fp16', 'float16')"

Decoding reads only that 44 MB branch; the full tokenizer is loaded when encoding a reference clip.

from omnivoice_mlx import OmniVoiceTTS

tts = OmniVoiceTTS("models/mlx-q8-fp16")
voice = tts.make_prompt("my-voice.wav", "exactly what that clip says.")
r = tts.generate("今天天气不错，我们出去走走吧。", voice)   # r.audio: float32 24 kHz, r.rtf

What is in here

file	what
model.safetensors	int8 g64 backbone, fp16 activations, output heads in fp32
config.json	upstream config plus dtype, head_dtype, quantization
tokenizer.json, tokenizer_config.json	the Qwen3 text tokenizer, unchanged from upstream

bf16, fp16 and fp32 builds are one minute of scripts/convert.py in the port's repository; only the recommended configuration is published here.

Licence

what	licence
these files	CC-BY-NC-4.0, inherited from k2-fsa/OmniVoice; non-commercial, attribution required
the conversion code	Apache-2.0, see omnivoice-mlx
Higgs codec weights	Boson Higgs Audio 2 Community License; not redistributed here, download from upstream as shown above