Spaces:
Runtime error
Runtime error
File size: 2,864 Bytes
72835fe |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
# vakyansh-tts
Text to Speech for Indic languages
## Pretrained Models
The pretrained models are hosted in a seperate repository [here](https://github.com/Open-Speech-EkStep/vakyansh-models#tts-models-repo)
### 1. Installation and Setup for training
Clone repo
```
git clone https://github.com/Open-Speech-EkStep/vakyansh-tts
```
Build conda virtual environment
```
cd ./vakyansh-tts
conda create --name <env_name> python=3.7
conda activate <env_name>
pip install -r requirement.txt
```
Install [apex](https://github.com/NVIDIA/apex); commit: 37cdaf4 for Mixed-precision training
```
cd ..
git clone https://github.com/NVIDIA/apex
cd apex
git checkout 37cdaf4
pip install -v --disable-pip-version-check --no-cache-dir ./
cd ../vakyansh-tts
```
Build Monotonic Alignment Search Code (Cython)
```
bash install.sh
```
### 1.1 Installation of tts_infer package
In tts_infer package, we currently have two components:
1. Transliteration (AI4bharat's open sourced models) (Languages supported: {'hi', 'gu', 'mr', 'bn', 'te', 'ta', 'kn', 'pa', 'gom', 'mai', 'ml', 'sd', 'si', 'ur'} )
2. Num to Word (Languages supported: {'en', 'hi', 'gu', 'mr', 'bn', 'te', 'ta', 'kn', 'or', 'pa'} )
```
git clone https://github.com/Open-Speech-EkStep/vakyansh-tts
cd vakyansh-tts
bash install.sh
python setup.py bdist_wheel
pip install -e .
cd tts_infer
gsutil -m cp -r gs://vakyaansh-open-models/translit_models .
```
Usage:
```
from tts_infer.tts import TextToMel, MelToWav
from tts_infer.transliterate import XlitEngine
from tts_infer.num_to_word_on_sent import normalize_nums
import re
from scipy.io.wavfile import write
text_to_mel = TextToMel(glow_model_dir='/path/to/glow-tts/checkpoint/dir', device='cuda')
mel_to_wav = MelToWav(hifi_model_dir='/path/to/hifi/checkpoint/dir', device='cuda')
def translit(text, lang):
reg = re.compile(r'[a-zA-Z]')
engine = XlitEngine(lang)
words = [engine.translit_word(word, topk=1)[lang][0] if reg.match(word) else word for word in text.split()]
updated_sent = ' '.join(words)
return updated_sent
def run_tts(text, lang):
text = text.replace('।', '.') # only for hindi models
text_num_to_word = normalize_nums(text, lang) # converting numbers to words in lang
text_num_to_word_and_transliterated = translit(text_num_to_word, lang) # transliterating english words to lang
mel = text_to_mel.generate_mel(text_num_to_word_and_transliterated)
audio, sr = mel_to_wav.generate_wav(mel)
write(filename='temp.wav', rate=sr, data=audio) # for saving wav file, if needed
return (sr, audio)
```
### 2. Spectogram Training (glow-tts)
```
cd ./scripts
bash train_glow.sh
```
### 3. Genrate Mels
```
cd ./scripts
bash generate_mels.sh
```
### 4. Vocoder Training (hifi-gan)
```
cd ./scripts
bash train_hifi.sh
```
### 4. Inference
```
cd ./scripts
bash infer.sh
```
|