wrice/chatterbox-turbo-grpo

Chatterbox Turbo, GRPO post-trained

Chatterbox Turbo whose text-to-speech-token model (T3) was post-trained with Group Relative Policy Optimization (GRPO) against rewards computed on decoded audio: intelligibility (Whisper WER and teacher-forced NLL), predicted quality (Distill-MOS), speaker similarity, and a duration guard. Only T3 changed, through a LoRA adapter that is merged into the weights here; S3Gen, the S3 tokenizer, the voice encoder, and the tokenizer are byte-identical to the base release.

Training code: github.com/will-rice/chatterbox-rl (commit 79c9222). Training run: wandb.

Results

200 held-out LibriTTS-R train-clean-100 prompts (speakers seen in training, sentences not), one sample per prompt at the Turbo default temperature 0.8, scored identically for this model and the base model at every validation. Numbers are the mean of the last eleven validations (steps 2,600 to 3,000); the base model's spread across those validations gives the noise floor (about 0.004 WER).

Metric	Base	This model	Result
WER, lower is better (Whisper large-v3-turbo)	0.045	0.036	20% fewer errors
Whisper teacher-forced NLL, lower is better	0.461	0.428	7% lower
Distill-MOS, higher is better	4.40	4.47	+0.07
Speaker similarity, CAMPPlus (reward model), higher is better	0.680	0.688	+0.009
Speaker similarity, WavLM-base-plus-sv (independent), higher is better	0.939	0.940	+0.001
log-F0 standard deviation (prosody guard)	0.234	0.240	unchanged
Words per second (pace guard)	2.728	2.727	unchanged
Rollouts without EOS (runaway guard)	0%	0%	unchanged

The last four rows are guards against the failure modes reported for RL-trained TTS: monotone prosody, voice drift toward the reward model, slower speech, and runaway generation. None moved.

Same sentence, same seed, both models.

Base:

This model:

Files: samples/base.wav, samples/grpo.wav.

Usage

Loads with the chatterbox-tts package as a drop-in for the base checkpoint:

from huggingface_hub import snapshot_download
from chatterbox.tts_turbo import ChatterboxTurboTTS

path = snapshot_download("wrice/chatterbox-turbo-grpo")
tts = ChatterboxTurboTTS.from_local(path, "cuda")
wav = tts.generate("Hello from the post-trained model.", audio_prompt_path="voice.wav")

adapter/ holds the unmerged PEFT LoRA (r=32, alpha=64 on c_attn, c_proj, c_fc) for anyone who prefers to apply it to the base T3 themselves or to continue training with the repository above.

Training

• Algorithm: GRPO, on-policy with one optimizer step per rollout batch, token-level loss aggregation, group-mean advantages without std division, k3 KL penalty (β = 0.02) to the LoRA-disabled base model.
• Rollouts: 8 prompts × 8 samples per step, temperature 1.0, cap 800 speech tokens (32 s); 3,000 steps, so about 24,000 prompts seen once.
• Rewards (each mapped to [0, 1], standardized across the batch, then weighted): intelligibility 0.4 (weighted harmonic mean of 1 − tanh(3·WER) and exp(−NLL/3) from Whisper large-v3-turbo), Distill-MOS 0.4, CAMPPlus speaker cosine 0.1, duration 0.1 (1 − |log(actual/expected)| / log 1.5, expected from the reference speaker's characters per second). Rollouts that never emit EOS score 0 on every term.
• Policy: LoRA r=32, alpha=64, learning rate 3e-5, AdamW, 20-step warmup.
• Data: LibriTTS-R train-clean-100, 32,947 sentences over 237 speakers; one 6 to 15 s utterance per speaker is the voice prompt, the rest are text prompts. No target audio is used.
• Hardware: one RTX 6000 Ada, about 25 s per step.

Limitations

• English only, trained on read audiobook speech; behaviour on paralinguistic tags such as [laugh] was not trained or evaluated.
• Gains are measured by automatic metrics from the same model families used for the rewards; no listening test was run.
• Speaker similarity had no headroom to gain: on real LibriTTS-R recordings, two utterances of the same speaker score a CAMPPlus cosine of 0.68 (sd 0.22, 60 pairs; different speakers 0.43), and both the base model (0.680) and this model (0.688) already match that level, so the similarity reward acted as a guard against drift rather than a driver.
• On individual sentences the model can pace differently from the base (the demo clip is about 7% slower); the average speaking rate over the held-out set is unchanged.

License

MIT, matching the base model's code and weights.