avraham123/tale-me-chatterbox-v3-browser

Chatterbox Multilingual V3 for Tale Me browsers

This repository contains a browser-oriented ONNX conversion of the official ResembleAI/chatterbox Multilingual V3 checkpoint. It is intended for local inference in the Tale Me application through a pinned Transformers.js 4.2.0 browser runtime with WebGPU.

The included default_speaker.safetensors is the built-in voice conditioning distributed by Resemble AI in the official conds.pt. It is not a recording uploaded by a Tale Me user. The browser does not need to upload or process a voice sample.

License and provenance

Chatterbox and its official model weights are distributed under the MIT License; the complete text is in LICENSE. The mandatory audio watermarker is an independent ONNX export of Resemble AI's MIT-licensed Perth source; its license is in LICENSE-PERTH.

Pinned sources:

• Official model: ResembleAI/chatterbox at 5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18
• Official source code: resemble-ai/chatterbox at 5de7a54aa4e5e2baadb0182dde554908b48b85c2
• Multilingual V3 checkpoint SHA-256: 5abca8321ede76f8e61f1cc0d19aea6c946b28871017ce8726f8a69203f05953
• Official built-in conditions SHA-256: 6552d70568833628ba019c6b03459e77fe71ca197d5c560cef9411bee9d87f4e
• Perth source commit: f83052aa42a0a47b9b62ff041c6a9332945fdee4
• Licensed ONNX graph base for the embedding and decoder structure: onnx-community/Chatterbox-ONNX at 452d3f434aa592098f1eedac9099f33642ab2da5 (MIT). The T3 embedding and language-model weights in this repository were replaced with the pinned V3 weights.

No endorsement by Resemble AI, Hugging Face, Microsoft, or the ONNX Community is implied.

Browser artifacts

• embed_tokens: official V3 text/speech/position/emotion embeddings, FP32.
• language_model_q4f16: official V3 T3 transformer and speech head, symmetric INT4 block-32 weights with FP16 compute and FP32 embedding/logit boundaries.
• conditional_decoder: S3Gen flow and vocoder matching the official pinned s3gen.pt weights.
• default_speaker.safetensors: official built-in voice conditioning.
• implicit_watermarker_24000: independently exported Perth watermarker, applied to every returned 24 kHz waveform.

The exact file hashes, sizes, source revisions, conversion tools, and verification results are recorded in conversion-manifest.json.

Verification summary

• V3 embedding graph: exact FP32 parity for prefill and autoregressive inputs.
• V3 language model: ONNX Runtime 1.26 browser runtime smoke test passed with 62 inputs and 61 outputs; random-input FP32-vs-INT4 comparison preserved the argmax and 17 of the top 20 logits.
• Tokenizer: exact core-token parity with the official tokenizer for Hebrew samples, plus the required Chatterbox framing tokens.
• Perth: PyTorch-vs-ONNX maximum absolute error 1.31353736e-05 for a 24 kHz test waveform.
• Built-in speaker tensor contract: audio_features [1,33,1024], audio_tokens [1,157], speaker_embeddings [1,192], speaker_features [1,314,80].

The 33 audio_features vectors intentionally exclude the checkpoint's final emotion vector because embed_tokens supplies the requested exaggeration vector. Including both would duplicate emotion conditioning.

Runtime note

This conversion expects the Tale Me vendored Transformers.js Chatterbox runtime, which adds multilingual classifier-free guidance, Min-P sampling, official V3 waveform trimming, and deterministic tensor/KV-cache cleanup. Chatterbox synthesis requires a recent desktop browser with WebGPU and ample memory. Model files are downloaded directly into the user's browser cache.