Chatterbox-ONNX-WebGPU

ResembleAI/chatterbox (English) for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference: a smaller speech decoder and a precomputed default voice. The language model and token embeddings are onnx-community's, used unchanged from onnx-community/chatterbox-ONNX (revision 3cab09af): language_model_q4, embed_tokens and tokenizer.json.

Changes from the original

• onnx/conditional_decoder.onnx: onnx-community's float32 conditional_decoder.onnx quantized without calibration (round to nearest, symmetric, blocks of 32): matrix multiplications to 4-bit MatMulNBits, convolution weights to blocked int8 DequantizeLinear (opset 21; 4 bits made the vocoder brighter). Activations stay float32. 534 MB → 145 MB.
• voices/default.bin (0.26 MB): what onnx-community's speech_encoder.onnx (float32) outputs for its default_voice.wav, computed once, so the browser needs neither the speech encoder (591 MB) nor a reference clip. Little-endian float32: N, P, F, then audio_features [N, 1024], audio_tokens [P] (whole numbers), speaker_embeddings [192] and speaker_features [F, 80]. Made by tools/export/chatterbox_voice.py in Quick Inference.

The decoder is a modified file: its output is close to, but not identical to, the original's. On eight test sentences, Whisper small.en heard it with a 4.5% word error rate, the same as the original float32 graphs.

Use

As the ONNX example in onnx-community's model card, with this file's four arrays in place of the speech encoder's outputs (audio_features before the text's embeddings in the first language-model step, exaggeration 0.5, greedy speech tokens with repetition penalty 1.2 until token 6562) and this decoder: audio_tokens and the new tokens with the two speaker arrays → 24 kHz audio. Quick Inference speaks one sentence at a time and runs the text through the Python package's punc_norm first.

Licence

MIT, as Chatterbox (LICENSE).