Chatterbox-Turbo-ONNX-WebGPU

The default voice for running ResembleAI/chatterbox-turbo in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference. The model itself is Resemble AI's own ONNX export, used unchanged from ResembleAI/chatterbox-turbo-ONNX (revision d21799bd): its int4 embed_tokens_q4, language_model_q4 and conditional_decoder_q4 graphs and tokenizer.json.

Contents

voices/default.bin (0.7 MB): what that repo's speech_encoder.onnx (float32) outputs for Chatterbox's default voice clip (default_voice.wav of onnx-community/chatterbox-ONNX), computed once, so the browser needs neither the speech encoder (230 MB at int4) nor a reference clip. Little-endian float32: N, P, F, then audio_features [N, 1024], audio_tokens [P] (whole numbers), speaker_embeddings [192] and speaker_features [F, 80]. Made by tools/export/chatterbox_voice.py in Quick Inference.

Use

As the ONNX example in Resemble AI's model card, with this file's four arrays in place of the speech encoder's outputs: audio_features before the text's embeddings in the first language-model step, then greedy speech tokens (repetition penalty 1.2) until token 6562; the decoder takes audio_tokens, the new tokens and three silence tokens (4299) with the two speaker arrays, and returns 24 kHz audio. Quick Inference speaks one sentence at a time and runs the text through the Python package's punc_norm first.

On eight test sentences, Whisper small.en heard it with a 7.95% word error rate, the same as the original float32 graphs with the clip itself.

Licence

MIT, as Chatterbox (LICENSE).