DACVAE-TTS English: every A/B arm in one place

Training on Vyvo/en-podcast-1, code and write-up: https://github.com/kadirnar/dacvae-tts. One folder per run: checkpoints/step-*.pt, eval/step-*/ (scores, per-sentence Whisper transcripts, audio), train.jsonl, config.json, its own README, RESULT.md (question, result against its baseline, verdict) and logs/.

EXPERIMENTS.md: the result log of every experiment (question, evidence, paired result against its baseline, trajectory, verdict and why).

Best model

full-base-s42 at 60,000 updates (checkpoint): 67.4M parameters, configs/en_base.yaml, trained only on en-podcast-1 (120 h after filtering). Try it in the demo.

Seed-TTS test-en DEV half (545 sentences, seeds 42 + 1000) WER % CER % SIM-o UTMOS22 DNSMOS
full-base-s42, 60k 2.63 1.29 0.599 2.995 2.535
DACVAE round trip of the ground truth (ceiling) 2.14 0.84 0.721 3.481 3.010

Listen: zero-shot voice cloning

20 clean reference voices that no training data contains (14 male, the rest female; EchoTTS speakers from SynDataLab-EN-Refs/echo-ref-speakers-4k-en, 44.1 kHz, chosen for the highest DNSMOS), each with its transcript, and a different sentence in each voice, synthesized by the snapshot at its operating point (32 Euler steps, guidance 5, prompt-rate duration). Single samples, seed 42, no reranking.

full-base-s42 at 60,000 updates

# voice sentence reference voice generated
1 male, 110 Hz, ref DNSMOS 3.63 Are they twins or just normal siblings?
2 male, 103 Hz, ref DNSMOS 3.55 He must be disguised to avoid encounters with thieves.
3 female, 187 Hz, ref DNSMOS 3.53 The town of Norwich is located east of the center of the county.
4 male, 142 Hz, ref DNSMOS 3.61 Constrictor knots can also be quite effective as improvised hose clamps or cable ties.
5 male, 128 Hz, ref DNSMOS 3.55 These two agencies co-operate in aspects of law enforcement, including their joint marine unit.
6 female, 173 Hz, ref DNSMOS 3.63 It then became part of Poland.
7 male, 102 Hz, ref DNSMOS 3.60 Will you be happy when I am in the earth?
8 male, 121 Hz, ref DNSMOS 3.60 She was involved in several record productions.
9 female, 186 Hz, ref DNSMOS 3.61 The Larson Brothers Airport is located in the town.
10 male, 116 Hz, ref DNSMOS 3.60 Flood gates were erected and drainage systems improved.
11 male, 119 Hz, ref DNSMOS 3.59 He did not live to complete the development of his system.
12 female, 190 Hz, ref DNSMOS 3.54 The land is volcanic in origin with mountains in the interior.
13 male, 141 Hz, ref DNSMOS 3.53 The Tuesday market is held by the blue bridge over the River Iro.
14 male, 96 Hz, ref DNSMOS 3.56 The evidence against Strafford was weak, and the impeachment failed.
15 female, 188 Hz, ref DNSMOS 3.57 It is operated by the New York City Department of Parks and Recreation.
16 male, 128 Hz, ref DNSMOS 3.57 Also that year, he collaborated with Evan Dando on a new Lemonheads album.
17 male, 129 Hz, ref DNSMOS 3.53 Finally, lumping groups of one's own pieces together is usually poor strategy.
18 female, 228 Hz, ref DNSMOS 3.59 Over time the couple has four more children, although William never warms to them.
19 male, 139 Hz, ref DNSMOS 3.54 The battle was of medium term significance, consolidating Royalist control of Yorkshire.
20 male, 99 Hz, ref DNSMOS 3.56 Before guns were invented, armies had to throw bullets at each other and if a bullet touched you, you had to sit out until the next war.

Everything else

Every run, checkpoint and evaluation is in the repository files, one folder per run (base-s42, base-s43, continue60k-lowlr-s42, en-baseline, en-corpus-build, en-duration, full-base-s42, liam-ft-base60k-s42, liam-posttrain-20260928, repa-hubert-large, repa-off), and every experiment's result is in EXPERIMENTS.md.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support