VoiceClone — F5-TTS fine-tune

Zero-shot voice cloning: give it a 6–15 s reference clip and arbitrary text, and it synthesizes that text in the reference speaker's voice. Fine-tuned from SWivid/F5-TTS (F5TTS_v1_Base, ~336M params, 24 kHz, Vocos vocoder).

Non-commercial. The base weights inherit CC-BY-NC-4.0 from the Emilia corpus, and that restriction flows through to this fine-tune. See Licensing below.

Current checkpoint

Training step	12000
Precision	bf16, full fine-tune (no LoRA)
Sample rate	24000 Hz

Evaluation

Measured on held-out speakers never seen during training, with sentences never seen during training — so these numbers reflect zero-shot generalisation, not memorisation.

Metric	Value	What it means
SECS ↑	0.7937 (+0.0067 ✅ vs base)	Speaker-encoder cosine similarity (ECAPA-TDNN). The primary "is cloning working" metric.
WER ↓	1.70% (+0.0000 ⚠️ vs base)	Whisper transcription vs input text. Catches intelligibility collapse.
CER ↓	1.46%	Character error rate.
UTMOS ↑	4.226	Predicted mean opinion score (naturalness).
RTF ↓	0.291	Real-time factor on an RTX 5070 Ti.

MCD is tracked internally but deliberately not published here: it is computed from MFCC rather than WORLD/SPTK mel-cepstrum, so it is valid for tracking this model's own progress but is not comparable to MCD figures in the literature.

Samples

• samples/000_libritts_r_2053.wav
• samples/001_libritts_r_2053.wav
• samples/002_libritts_r_2053.wav
• samples/003_libritts_r_2053.wav
• samples/004_libritts_r_2053.wav
• samples/005_vctk_p283.wav
• samples/006_vctk_p283.wav
• samples/007_vctk_p283.wav

Usage

from f5_tts.api import F5TTS

tts = F5TTS(model="F5TTS_v1_Base", ckpt_file="model.safetensors")
wav, sr, _ = tts.infer(
    ref_file="reference.wav",
    ref_text="Transcript of the reference clip.",
    gen_text="Text you want spoken in that voice.",
    file_wave="out.wav",
)

Reference clips should be 6–15 s, clean, and single-speaker. Quality of the reference dominates output quality far more than the length does.

Training data

Source	HF id	License	Attribution
libritts_r	mythicinfinity/libritts_r	CC BY 4.0	LibriTTS-R (Koizumi et al., 2023), derived from LibriTTS / LibriVox.
vctk	sanchit-gandhi/vctk	CC BY 4.0 (upstream CSTR VCTK; mirror declares none)	CSTR VCTK Corpus, University of Edinburgh (CC BY 4.0). Accessed via the sanchit-gandhi/vctk parquet mirror.

Audio was resampled to 24000 Hz mono, loudness-normalised to -23.0 LUFS (EBU R128), VAD-trimmed, and filtered on duration (1.0–15.0 s), estimated SNR (≥ 12.0 dB), clipping, and transcript plausibility.

Licensing

• This model: cc-by-nc-4.0 — non-commercial use only.
• Base weights (SWivid/F5-TTS) are CC-BY-NC-4.0 because the Emilia training corpus is CC-BY-NC. That restriction is viral onto any fine-tune, including this one.
• F5-TTS code is MIT.
• LibriTTS-R and VCTK are CC BY 4.0 and require attribution, given above.

For a commercially usable lineage you would need to retrain from an Apache-2.0 base such as mrfakename/OpenF5-TTS-Base (identical architecture) — fine-tuning this checkpoint cannot remove the NC restriction.

Ethical use

Voice cloning can be used to impersonate people. Only clone voices you have explicit permission to clone. Do not use this model for fraud, harassment, defamation, or to fabricate statements by real people.

---

Auto-generated by hf_push.py at 2026-07-27 04:31:46.