Sunbird AI Text-to-Speech (TTS) model trained on Luganda text
Text-to-Speech (TTS) with Tacotron2 trained on Female Luganda Commonvoice Recordings
This repository provides all the necessary tools for Text-to-Speech (TTS) with SpeechBrain.
The pre-trained model takes in input a short text and produces a spectrogram in output. One can get the final waveform by applying a vocoder (e.g., HiFIGAN) on top of the generated spectrogram.
Install SpeechBrain
pip install speechbrain
Please notice that we encourage you to read our tutorials and learn more about SpeechBrain.
Perform Text-to-Speech (TTS)
import torchaudio
from speechbrain.pretrained import Tacotron2
from speechbrain.pretrained import HIFIGAN
# Intialize TTS (tacotron2) and Vocoder (HiFIGAN)
tacotron2 = Tacotron2.from_hparams(source="sunbird/sunbird-lug-tts-commonvoice-female/", savedir="tmpdir_tts")
hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-ljspeech", savedir="tmpdir_vocoder")
# Running the TTS
mel_output, mel_length, alignment = tacotron2.encode_text("Mbagaliza Christmass Enungi Nomwaka Omugya Gubaberere Gwamirembe")
# Running Vocoder (spectrogram-to-waveform)
waveforms = hifi_gan.decode_batch(mel_output)
# Save the waverform
torchaudio.save('example_TTS.wav',waveforms.squeeze(1), 22050)
If you want to generate multiple sentences in one-shot, you can do in this way:
from speechbrain.pretrained import Tacotron2
tacotron2 = Tacotron2.from_hparams(source="speechbrain/TTS_Tacotron2", savedir="tmpdir")
items = [
"Nsanyuse okukulaba",
"Erinnya lyo ggwe ani?",
"Mbagaliza Christmass Enungi Nomwaka Omugya Gubaberere Gwamirembe"
]
mel_outputs, mel_lengths, alignments = tacotron2.encode_batch(items)
Inference on GPU
To perform inference on the GPU, add run_opts={"device":"cuda"}
when calling the from_hparams
method.
- Downloads last month
- 12
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.