thealper2/speecht5-tts-turkish

Turkish text-to-speech, fine-tuned from microsoft/speecht5_tts on erenfazlioglu/turkishvoicedataset.

Paired with the microsoft/speecht5_hifigan vocoder it converts Turkish text into a 16 kHz waveform.

Model description

Task	Text-to-speech (Turkish)
Base model	microsoft/speecht5_tts (144M)
Vocoder	microsoft/speecht5_hifigan
Speaker encoder	speechbrain/spkrec-xvect-voxceleb (512-d x-vector)
Sampling rate	16,000 Hz
Voice	Single synthetic voice inherited from the training corpus

The vocabulary had to be extended

The base checkpoint ships a 79-piece character SentencePiece model built for English. It contains none of ç ğ ı İ ö ş ü and no digits, so 99.55% of the Turkish corpus contained at least one character it could not represent.

This model therefore uses an extended tokenizer (99 tokens): the Turkish letters were appended as real SentencePiece pieces, and the text embedding was grown from 81 to 99 rows with every pre-trained vector carried over and each new letter seeded from a related base letter.

The processor bundled with this repository is required. Loading these weights with the base microsoft/speecht5_tts processor will produce wrong output.

Numbers must still be spelled out — the vocabulary has no digits. The inference code does this automatically; if you call the model directly, expand numbers to Turkish words first (2026 → iki bin yirmi altı).

Usage

import numpy as np
import soundfile as sf
import torch
from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor

repo = "thealper2/speecht5-tts-turkish"
processor = SpeechT5Processor.from_pretrained(repo)
model = SpeechT5ForTextToSpeech.from_pretrained(repo)
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")

# The reference speaker embedding is shipped with this repository.
from huggingface_hub import hf_hub_download

speaker = np.load(hf_hub_download(repo, "speaker_embedding.npy"))
speaker = torch.from_numpy(speaker).unsqueeze(0)

text = "Merhaba, bu Türkçe SpeechT5 modelinin test cümlesidir."
inputs = processor(text=text, return_tensors="pt")
speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)
sf.write("output.wav", speech.numpy(), 16000)

For number handling, punctuation normalisation and long-text chunking, use the repository's src/inference.py helper instead of calling the model directly.

Training data

Source	erenfazlioglu/turkishvoicedataset
Rows	130,634 (single train split)
Audio	MP3, 24 kHz mono, constant 48 kbps, resampled to 16 kHz
Total duration	279.2 h
Origin	Generated with Microsoft Text-to-Speech — synthetic, not recorded

The corpus is a single synthetic voice. Measured across 1,500 random clips, RMS standard deviation is 0.008 and x-vector cosine similarity to the corpus centroid averages 0.991 (minimum 0.940).

Split (deterministic, seed 42, duplicate-group aware so no near-duplicate family straddles a boundary): 117,625 train / 3,091 validation / 3,108 test, verified at 0 cross-split transcription collisions.

Training procedure

Hyperparameter	Value
Learning rate	2e-05
Schedule	linear, 441 (6%) warmup steps
Per-device batch size	16
Gradient accumulation	2
Effective batch size	32
Epochs	3
Optimiser steps	7,351
Weight decay	0.01
Precision	bf16
Gradient checkpointing	enabled
Guided attention loss	enabled
Layerdrop	0.0 (pre-trained default of 0.1 disabled for fine-tuning stability)
GPU	NVIDIA GeForce RTX 5060 Ti (7.96 GB)
Peak VRAM	4.859 GiB
Training time	63 min

Preprocessing: silence trimmed at 45 dB, audio capped at 12 s, text capped at 450 tokens (rows exceeding the cap are dropped rather than truncated), numbers expanded to Turkish words, typographic punctuation mapped onto the tokenizer's ASCII inventory.

Evaluation

Objective scoring uses text → this model → openai/whisper-small → text, over a fixed 30-sentence Turkish set covering short, medium, long, Turkish-character-heavy, numeric and punctuation-heavy inputs.

Metric	Value
Best checkpoint	checkpoint-7000
Validation loss	0.29169
WER (fine-tuned)	0.1538
CER (fine-tuned)	0.0528
WER (base speecht5_tts)	1.0077
CER (base speecht5_tts)	0.4698
ASR error floor on real dataset audio	WER 0.1584 / CER 0.0398

Per-category breakdown of the fixed evaluation set:

Category	WER	CER	n
long	0.1333	0.0464	4
medium	0.05	0.0221	5
numbers	0.0833	0.0531	5
punctuation	0.1	0.0262	5
short	0.0	0.0	5
turkish_chars	0.4878	0.1312	6

Audio health across the same set: mean 4.07 s, longest internal gap 2.23 s, 0 silent outputs.

How to read these numbers

ASR error rates are auxiliary. The recogniser has its own error floor on this corpus (reported above); scores should be read against that floor, not against zero. An ASR mistake is not proof of a TTS mistake.

The fine-tuned WER landing close to the ASR floor does not mean the synthesis is as good as the original recordings. The two figures are measured on different text: the floor comes from real corpus sentences (news copy, dense with proper nouns), while the evaluation set is 30 hand-written sentences chosen to probe specific behaviours. Treat the floor as a scale marker, not as a target that has been beaten.

Some remaining errors are scoring artefacts rather than pronunciation faults — the recogniser writes cm for a spoken santimetre, and drops the circumflex in rüzgâr and zekâ. The true error rate is therefore somewhat lower than the table shows.

No MOS score is reported. No MOS study was conducted. Any subjective ratings in the project report are informal listening notes by a single listener, labelled as such.

Limitations

• Turkish-specific letters are the weakest point. The turkish_chars category scores roughly four times worse than every other category. Errors concentrate on ğ (Iğdır → idlir, öğleden → o neden) and on word-initial ş (şoförüyle → sofori). This is expected: those characters were added to the vocabulary for this fine-tune and had two epochs to train, against the base alphabet's full pre-training. ğ is additionally the hardest case in Turkish for any TTS, since it marks vowel lengthening rather than a segment of its own. More epochs, or oversampling ğ-dense sentences, is the obvious next experiment.
• Synthetic voice ceiling. The training audio was produced by a TTS engine, so this model reproduces that engine's timbre, speaking rate and prosody. It will not sound like a recorded human speaker, and the gap is a property of the data, not of the training run.
• Single speaker. No voice cloning or multi-speaker control. Substituting a different x-vector is out of distribution and degrades quality.
• Numbers and abbreviations must be expanded before tokenisation; the vocabulary has no digits.
• Domain skew. The corpus is predominantly Turkish news text, including politically charged material. Prosody is tuned to news-reading register.
• Long input. Quality degrades on very long single utterances; chunk on sentence boundaries.
• Non-Turkish text is out of scope — the tokenizer covers the Turkish alphabet plus basic Latin.

License and intended use

The training dataset is licensed CC BY-NC 4.0 (non-commercial), and fine-tuning does not remove that restriction. This repository therefore does not claim the weights are suitable for commercial use.

Before redistributing or using this model commercially, review:

1. the erenfazlioglu/turkishvoicedataset licence (CC BY-NC 4.0),
2. Microsoft's terms for Text-to-Speech output, since the training audio was generated by that service,
3. the MIT licence of the microsoft/speecht5_tts base model.

Intended for research and educational use.

Citation

@inproceedings{ao2022speecht5,
  title     = {SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing},
  author    = {Ao, Junyi and Wang, Rui and Zhou, Long and Wang, Chengyi and Ren, Shuo and Wu, Yu
               and Liu, Shujie and Ko, Tom and Li, Qing and Zhang, Yu and Wei, Zhihua and Qian, Yao
               and Li, Jinyu and Wei, Furu},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics},
  year      = {2022}
}