owensong/chatterbox-nano-ONNX

Chatterbox Nano ONNX

Community ONNX conversion of ResembleAI/chatterbox-nano, packaged for ONNX Runtime and custom browser applications using ONNX Runtime Web.

This is an independent conversion. It is not an official Resemble AI release.

Status

The four-graph ONNX pipeline has run end to end in local tests. Evaluation is currently limited to development smoke tests, not a representative speech-quality benchmark.

This repository does not publish WER, PESQ, STOI, speaker-similarity, MOS, latency, or real-time-factor claims. We have not evaluated enough speakers, prompts, reference recordings, devices, or random seeds to report those metrics responsibly.

The package is non-streaming. It generates speech tokens autoregressively and then synthesizes the complete waveform in one acoustic-decoder call. Experimental chunked-streaming graphs are intentionally excluded.

What Has Been Tested

• All four packaged ONNX graphs load through ONNX Runtime in the development environment.
• External tensor-data references resolve from the published file layout.
• An end-to-end CPU smoke test accepted one reference recording and one English sentence and produced a playable 24 kHz mono WAV.
• The mixed-precision graphs loaded and produced audio through ONNX Runtime Web/WebGPU in Chrome on one Windows machine with an RTX 3060.
• Autoregressive language-model execution was exercised with an initially empty and then growing KV cache.

These checks establish that the conversion can execute. They do not establish general speech quality, speaker similarity, robustness, or performance.

What Has Not Been Established

• No benchmark dataset or statistically meaningful evaluation set has been run.
• No human mean-opinion-score study has been run.
• Intelligibility has not been measured across diverse text, accents, speakers, microphones, or recording conditions.
• Voice similarity has not been measured across a representative speaker set.
• Long-form stability and paralinguistic tags have not been systematically evaluated in this ONNX conversion.
• CUDA, WebGPU, browser, and operating-system compatibility have not been tested across a device matrix.
• Production reliability, concurrency, memory limits, and adversarial inputs have not been evaluated.

Treat this repository as a functional conversion that still needs broader evaluation before production use.

Package Contents

The model is split into four ONNX sessions:

Stage	Graph	Published profile
Text/speech-token embedding	onnx/embed_tokens_fp16.onnx	FP16 embedding weights
Reference speech encoder	onnx/speech_encoder_q4f16.onnx	INT4 weight quantization with FP16 activations
Autoregressive speech-token model	onnx/language_model_q4f16.onnx	INT4 weight quantization with FP16 activations and KV cache
Acoustic decoder and vocoder	onnx/conditional_decoder_q4.onnx	INT4 weight quantization with FP32 activations

Each large graph uses an external tensor-data file. Keep every .onnx_data file beside its corresponding .onnx file. The complete repository is approximately 547 MiB.

The quantization labels describe how these files were produced. They are not quality claims and do not imply equivalent output to the original checkpoint for every input.

Architecture Flow

1. The speech encoder converts a reference recording into conditioning features, prompt speech tokens, and speaker representations.
2. The embedding graph converts text and generated speech-token IDs into model embeddings.
3. The language model autoregressively generates speech tokens while updating a 12-layer KV cache.
4. The conditional decoder consumes the prompt and generated speech tokens and returns a complete 24 kHz waveform.

Special token IDs used by the included runner:

Purpose	Token ID
Start speech	6561
Stop speech	6562
Decoder silence padding	4299

Download

Using the Hugging Face CLI:

hf download owensong/chatterbox-nano-ONNX \
  --local-dir chatterbox-nano-ONNX

Using Python:

from huggingface_hub import snapshot_download

model_dir = snapshot_download(
    repo_id="owensong/chatterbox-nano-ONNX",
    local_dir="chatterbox-nano-ONNX",
)
print(model_dir)

Use SHA256SUMS to verify downloaded files when reproducibility matters.

Python Example

The included run_onnx.py demonstrates the complete four-session pipeline. It is a reference implementation, not a production inference server.

Create an environment:

python -m venv .venv

Activate it and install the pinned smoke-test dependencies:

# Windows PowerShell
.venv\Scripts\Activate.ps1
pip install -r requirements.txt

# Linux or macOS
source .venv/bin/activate
pip install -r requirements.txt

Provide a voice reference you are authorized to use, then run:

python run_onnx.py \
  --model . \
  --voice reference.wav \
  --output output.wav \
  --text "Hello. This is Chatterbox Nano running through ONNX Runtime." \
  --dtype q4f16 \
  --provider cpu \
  --max-new-tokens 256 \
  --seed 1337

The runner resamples the reference to 24 kHz, executes the four graphs, and writes a mono WAV. Generated output varies because speech-token generation samples from the model unless --greedy is supplied.

For CUDA, install a version of onnxruntime-gpu compatible with your CUDA environment instead of the CPU-only onnxruntime dependency. CUDA execution of this published package has not been systematically validated.

Browser and WebGPU Use

These are raw ONNX component graphs, not a self-contained web application. A browser integration must provide:

• Reference-audio decoding and resampling.
• Four ONNX Runtime Web sessions.
• Tokenization and autoregressive sampling.
• KV-cache allocation and updates.
• Waveform conversion and playback.
• Cancellation, error handling, and memory management.

The repository is not directly compatible with a generic pipeline() call. ONNX Runtime Web operator support also varies by browser and GPU. Successful execution on one RTX 3060 system should not be interpreted as a compatibility or speed guarantee for other hardware.

Non-Streaming Behavior

Chatterbox Nano's published acoustic path is not an incremental streaming decoder. This conversion therefore waits for speech-token generation and performs a full acoustic synthesis pass.

Do not market this package as streaming or suitable for low-latency voice-agent use without replacing or retraining the acoustic path and validating the resulting model. Concatenating independently decoded chunks can introduce pauses, discontinuities, and inconsistent prosody.

Upstream Sources

• Base model: ResembleAI/chatterbox-nano
• Official ONNX reference used during conversion: ResembleAI/chatterbox-turbo-ONNX
• Original implementation: resemble-ai/chatterbox

The conversion process used the official Turbo ONNX package as a graph reference for shared speech components and applied the Nano-specific dimensions and conditioning weights required by this checkpoint.

Upstream Chatterbox documentation describes Nano as an English text-to-speech model with voice conditioning and paralinguistic tags. Those upstream capabilities should not be confused with independent validation of this ONNX conversion.

Responsible Use

Only synthesize or clone a person's voice with authorization. Users are responsible for complying with applicable privacy, publicity, copyright, impersonation, disclosure, and consent requirements. Synthetic audio should not be represented as authentic human speech when doing so could mislead listeners.

License and Attribution

The upstream Chatterbox project and model are released under the MIT License. See LICENSE and NOTICE.md. Chatterbox and the original model weights are attributed to Resemble AI. This repository provides independent ONNX conversion and packaging work.