DACVAE-TTS Turkish: every tr-combined experiment in one place
Training on Codyfederer/tr-combined, code and write-up: https://github.com/kadirnar/dacvae-tts (branch trc/tr-combined-experiments). One folder per run: checkpoints/step-*.pt, eval/step-*/ (scores, per-sentence Whisper transcripts, audio), train.jsonl, config.json, its own README, RESULT.md (question, result against its baseline, verdict) and logs/.
EXPERIMENTS.md: the result log of every experiment (question, evidence, paired result against its baseline, trajectory, verdict and why).
Current result: new models vs run C (single sample, no reranking)
Same inference for all three: duration predictor refit on tr-combined, guidance 5, 32 steps, 495 Freya-TR-Eval sentences × 48 leak-free Common Voice voices, sampling seeds 42 + 1000 pooled. Brackets: paired speaker-clustered jackknife-t 95 % interval of the difference to run C.
| model | WER % | CER % | SIM-o | DNSMOS | UTMOS |
|---|---|---|---|---|---|
run C (VoiceHub/dacvae-tts-tr-w512, old data, 70 h) |
5.10 | 2.93 | 0.519 | 2.860 | 2.493 |
full-cross: tr-combined + cross-utterance prompts, 60k |
2.94 [-3.27, -1.05] | 1.66 [-2.18, -0.34] | 0.536 | 2.936 | 2.572 |
full-v2: + quality condition + speech-REPA + character units, 60k |
0.93 [-5.35, -2.99] | 0.36 [-3.52, -1.62] | 0.556 | 3.128 | 2.533 |
full-v2 against full-cross: WER -2.01 [-2.55, -1.46], CER -1.31 [-1.66, -0.95], SIM-o +0.021 [0.010, 0.031],
DNSMOS +0.192 [0.163, 0.221], UTMOS -0.039 [-0.084, 0.006] (tie). Seed 42: 468 of 495 sentences have no word error; the rest
are near-homophones ("Vurmak da" → "Vurmakta"), foreign names and one repeated phrase.
What was adopted, from the 20k A/B arms below: cross-utterance prompts (#11), speech-REPA (#10: WER halved, alignment
~4x earlier), the quality condition (DNSMOS control), character units (small, consistent over two training seeds).
Rejected: latent negatives (#8, collapse), TLA-SA (#10), model-guidance fine-tune (#14), Flow-GRPO at 600 updates
(#16, no effect). Side-by-side audio: comparison.
Inference comparisons: systems.
Final evaluation of every run
Freya-TR-Eval (495 sentences) spoken by 48 leak-free Common Voice test voices, one sample per sentence (no reranking), guidance 5, 32 Euler steps, prompt-rate duration rule, Whisper large-v3, turkish-v2 metric. Seeds: sampling seeds of the final evaluation (2 = 42 and 1000 pooled). A/B arms stop at 20k of the 60k schedule; training-seed spread there is ~5 WER points.
| run | what | updates | seeds | WER % | CER % | SIM-o | DNSMOS | UTMOS |
|---|---|---|---|---|---|---|---|---|
full-cross |
model candidate: run C recipe + cross prompts, 60k | 60000 | 2 | 6.46 | 4.40 | 0.548 | 2.970 | 2.519 |
full-v2 |
model candidate v2: cross + quality + REPA + char units, 60k: quality_condition=true, quality_scores=quality/dnsmos.json, quality_target=[4.0, 4.5, 3.8], re... |
60000 | 2 | 2.21 | 1.17 | 0.577 | 3.164 | 2.515 |
run-c-reference |
run C (VoiceHub/dacvae-tts-tr-w512, old data Vyvo/tr-dataset-12), same protocol | 60000 | 1 | 7.54 | 4.89 | 0.535 | 2.903 | 2.420 |
y-base |
20000 | 2 | 4.32 | 2.98 | 0.587 | 3.111 | 2.486 | |
y-cond-text-pool |
#9, on v2: cond_text_pool=true |
20000 | 2 | 4.53 | 3.07 | 0.590 | 3.112 | 2.499 |
y-decay-matrices |
audit: weight decay on matrices only, on v2: weight_decay_scope=matrices |
20000 | 2 | 3.77 | 2.11 | 0.581 | 3.122 | 2.480 |
y-ffn-conv |
#9, on v2: ffn_conv_kernel=5 |
20000 | 2 | 4.04 | 2.74 | 0.598 | 3.090 | 2.415 |
y-final-adaln |
#9, on v2: final_adaln=true |
20000 | 2 | 4.04 | 2.27 | 0.588 | 3.128 | 2.495 |
y-long-skip |
#9, on v2: long_skip=true |
20000 | 2 | 3.99 | 2.76 | 0.589 | 3.110 | 2.512 |
y-regularized |
#14 dropout 0.1 + weight decay 0.05 on matrices, on v2: dropout=0.1, weight_decay=0.05, weight_decay_scope=matrices |
20000 | 2 | 4.58 | 3.05 | 0.572 | 3.043 | 2.397 |
y-s43 |
noise floor of the v2 recipe: seed=43 |
20000 | 2 | 3.64 | 2.30 | 0.571 | 3.112 | 2.506 |
y-speaker-condition |
round 2, on v2: speaker_condition_dim=192, speaker_condition=teacher/ecapa-speechbrain |
20000 | 2 | 3.66 | 2.40 | 0.599 | 3.039 | 2.295 |
y-value-residual |
#9, on v2: value_residual=true |
20000 | 2 | 3.94 | 2.29 | 0.582 | 3.138 | 2.533 |
ft-mg-w07 |
#14 model guidance w=0.7 (sampled at g=1) | 8000 | 2 | 9.01 | 5.82 | 0.526 | 3.022 | 2.546 |
ft-w0 |
#14 control: same fine-tune, w=0 (sampled at g=5) | 8000 | 2 | 6.58 | 4.42 | 0.546 | 2.964 | 2.486 |
grpo |
#16 Flow-GRPO from full-cross 60k | 600 | 2 | 6.70 | 4.51 | 0.551 | 2.976 | 2.536 |
base-eager |
#7 | 20000 | 2 | 20.12 | 13.12 | 0.553 | 2.972 | 2.430 |
base-s42 |
#7 baseline | 20000 | 2 | 30.09 | 22.11 | 0.516 | 2.925 | 2.312 |
base-s42-pad64 |
#7 baseline with lengths padded to multiples of 64 (20 % of every batch was padding) | 20000 | 2 | 27.87 | 17.76 | 0.534 | 2.886 | 2.303 |
base-s43 |
seed noise floor: seed=43 |
20000 | 2 | 24.90 | 15.02 | 0.543 | 2.884 | 2.329 |
pairs-cross |
#11: cross_prompt_prob=0.4, cross_prompt_max_utterances=3, cross_prompt_max_seconds=12.0 |
20000 | 2 | 12.30 | 8.25 | 0.540 | 2.961 | 2.445 |
x-attn-gate |
#9, on base+cross: attn_gate=head |
20000 | 2 | 16.67 | 10.25 | 0.529 | 2.917 | 2.375 |
x-char-units |
round 2, on base+cross: text_units=chars |
20000 | 2 | 11.17 | 7.41 | 0.551 | 2.973 | 2.439 |
x-char-units-s43 |
round 2 char units, training seed 43: text_units=chars, seed=43 |
20000 | 2 | 16.08 | 9.71 | 0.546 | 2.926 | 2.341 |
x-latent-negatives |
#8, on base+cross: contrastive_mode=latent_delta, contrastive_random_weight=0.2, contrastive_aug_weight=0.2, contrastive_span_min=3, contrastive_span_max=12... |
20000 | 2 | 62.15 | 38.60 | 0.139 | 1.919 | 1.258 |
x-long-skip |
#9, on base+cross: long_skip=true |
20000 | 2 | 13.01 | 8.21 | 0.535 | 2.953 | 2.407 |
x-no-negatives |
#8, on base+cross: contrastive_mode=none |
20000 | 2 | 13.92 | 9.90 | 0.540 | 2.965 | 2.430 |
x-pairs-char-ctc |
#11, on base+cross: ctc_targets=chars |
20000 | 2 | 13.64 | 8.73 | 0.540 | 2.939 | 2.429 |
x-pairs-tail |
#11, on base+cross: tail_silence_prob=0.3, tail_silence_max_seconds=0.8, long_prompt_prob=0.25, prompt_fraction_long_max=0.85, prompt_cut=quiet |
20000 | 2 | 13.26 | 8.89 | 0.543 | 2.972 | 2.438 |
x-quality-cond |
new: quality condition, on base+cross: quality_condition=true, quality_scores=quality/dnsmos.json |
20000 | 2 | 13.22 | 8.67 | 0.533 | 3.020 | 2.491 |
x-repa |
#10, on base+cross: repa_layer=10, repa_dim=256, teacher_features=teacher/mhubert147-l12-pca256, repa_weight=1.0, repa_stop_step=0, repa_frames=all |
20000 | 2 | 5.56 | 3.76 | 0.585 | 2.882 | 2.207 |
x-s43 |
noise floor of base+cross: seed=43 |
20000 | 2 | 16.94 | 9.66 | 0.533 | 2.936 | 2.394 |
x-swiglu |
#9, on base+cross: ffn_activation=swiglu |
20000 | 2 | 17.45 | 9.91 | 0.524 | 2.903 | 2.366 |
x-tla |
#10, on base+cross: tla_layers=all, tla_dim=192, tla_hidden=256, speaker_embeddings=teacher/ecapa-speechbrain, tla_weight=0.5, tla_entropy=0.01 |
20000 | 2 | 24.67 | 14.05 | 0.455 | 2.771 | 2.167 |
Training trajectories
WER / CER % per snapshot: the first 96 sentences (quick check) before the final step, seed 42.
| run | 5k | 10k | 15k | 20k | 25k | 30k | 35k | 40k | 45k | 50k | 55k | 60k |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
full-cross |
92.9 / 53.8 | 27.9 / 16.5 | 17.9 / 11.5 | 12.0 / 8.5 | 11.4 / 7.2 | 9.9 / 6.5 | 9.2 / 6.2 | 7.2 / 4.8 | 8.1 / 4.9 | 8.3 / 5.9 | 7.9 / 6.2 | 6.7 / 4.7 |
full-v2 |
11.3 / 6.7 | 6.8 / 4.9 | 5.8 / 4.5 | 4.3 / 3.7 | 5.1 / 3.3 | 3.4 / 2.4 | 2.4 / 1.6 | 2.6 / 1.8 | 2.3 / 1.7 | 2.2 / 1.3 | 1.8 / 1.1 | 2.4 / 1.4 |
y-base |
11.3 / 6.7 | 6.8 / 4.9 | 5.8 / 4.5 | 4.3 / 2.9 | ||||||||
y-cond-text-pool |
12.2 / 8.5 | 5.5 / 3.3 | 5.5 / 3.6 | 4.4 / 2.9 | ||||||||
y-decay-matrices |
12.3 / 7.7 | 7.4 / 5.1 | 5.0 / 3.0 | 3.7 / 2.0 | ||||||||
y-ffn-conv |
10.8 / 6.5 | 6.0 / 4.4 | 4.9 / 3.4 | 4.2 / 2.8 | ||||||||
y-final-adaln |
12.4 / 9.2 | 7.5 / 5.2 | 5.5 / 3.4 | 4.2 / 2.3 | ||||||||
y-long-skip |
12.3 / 8.9 | 7.3 / 5.2 | 5.5 / 3.4 | 3.9 / 2.9 | ||||||||
y-regularized |
10.7 / 6.7 | 4.4 / 3.5 | 6.4 / 4.7 | 4.8 / 3.2 | ||||||||
y-s43 |
11.4 / 6.3 | 7.5 / 4.7 | 4.8 / 3.5 | 3.6 / 2.3 | ||||||||
y-speaker-condition |
10.1 / 6.2 | 7.2 / 5.1 | 4.1 / 3.5 | 3.7 / 2.4 | ||||||||
y-swiglu |
12.6 / 7.7 | 7.1 / 5.1 | 6.5 / 4.4 | |||||||||
y-value-residual |
10.8 / 6.8 | 6.4 / 4.1 | 7.3 / 4.4 | 3.6 / 2.0 | ||||||||
base-eager |
73.3 / 51.6 | 40.0 / 25.9 | 28.7 / 19.7 | 19.8 / 12.7 | ||||||||
base-s42 |
76.4 / 46.8 | 36.4 / 21.4 | 32.2 / 24.5 | 30.3 / 22.1 | ||||||||
base-s42-pad64 |
83.6 / 48.8 | 52.7 / 32.7 | 40.5 / 25.2 | 27.6 / 17.1 | ||||||||
base-s43 |
46.0 / 27.3 | 31.5 / 18.4 | 24.5 / 15.0 | |||||||||
pairs-cross |
97.6 / 58.5 | 29.6 / 17.6 | 17.3 / 11.9 | 12.0 / 8.2 | ||||||||
x-attn-gate |
99.1 / 56.3 | 29.3 / 16.9 | 18.8 / 12.0 | 16.8 / 10.3 | ||||||||
x-char-units |
93.8 / 55.8 | 30.6 / 18.1 | 16.3 / 9.9 | 11.1 / 7.4 | ||||||||
x-char-units-s43 |
85.6 / 48.7 | 33.6 / 19.2 | 21.0 / 13.3 | 16.2 / 9.8 | ||||||||
x-latent-negatives |
104.2 / 86.4 | 95.0 / 64.7 | 75.1 / 48.2 | 61.9 / 38.8 | ||||||||
x-long-skip |
90.5 / 56.4 | 30.6 / 18.3 | 17.4 / 10.9 | 12.9 / 8.1 | ||||||||
x-no-negatives |
101.9 / 59.3 | 34.5 / 19.5 | 17.9 / 12.3 | 14.4 / 10.1 | ||||||||
x-pairs-char-ctc |
98.7 / 66.6 | 26.7 / 16.4 | 19.4 / 12.3 | 13.6 / 8.6 | ||||||||
x-pairs-tail |
95.4 / 59.3 | 28.0 / 16.4 | 17.5 / 11.8 | 13.0 / 8.9 | ||||||||
x-quality-cond |
97.8 / 67.9 | 29.3 / 17.3 | 18.5 / 11.6 | 13.6 / 8.9 | ||||||||
x-repa |
13.6 / 8.9 | 10.5 / 7.9 | 7.2 / 5.0 | 6.1 / 3.9 | ||||||||
x-s43 |
101.7 / 58.1 | 36.7 / 19.0 | 29.3 / 16.1 | 17.2 / 9.6 | ||||||||
x-swiglu |
98.6 / 58.1 | 38.7 / 22.2 | 24.7 / 19.3 | 17.8 / 10.0 | ||||||||
x-tla |
112.3 / 83.0 | 43.5 / 25.2 | 29.6 / 17.7 | 24.2 / 13.4 |