Package: tacotron
Tacotron with Location Relative AttentionA PyTorch implementation ofLocation-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis. Audio samples can be foundhere. Colab demo can be foundhere.Fig 1:Tacotron (with Dynamic Convolution Attention).Fig 2:Example Mel-spectrogram and attention plot.Quick StartEnsure you have Python 3.6 and PyTorch 1.7 or greater installed. Then install this package (along with theunivoc vocoder):pip install tacotron univocExample UsageimporttorchimportsoundfileassffromunivocimportVocoderfromtacotronimportload_cmudict,text_to_id,Tacotron# download pretrained weights for the vocoder (and optionally move to GPU)vocoder=Vocoder.from_pretrained("https://github.com/bshall/UniversalVocoding/releases/download/v0.2/univoc-ljspeech-7mtpaq.pt").cuda()# download pretrained weights for tacotron (and optionally move to GPU)tacotron=Tacotron.from_pretrained("https://github.com/bshall/Tacotron/releases/download/v0.1/tacotron-ljspeech-yspjx3.pt").cuda()# load cmudict and add pronunciation of PyTorchcmudict=load_cmudict()cmudict["PYTORCH"]="P AY1 T AO2 R CH"text="A PyTorch implementation of Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis."# convert text to phone idsx=torch.LongTensor(text_to_id(text,cmudict)).unsqueeze(0).cuda()# synthesize audiowithtorch.no_grad():mel,_=tacotron.generate(x)wav,sr=vocoder.generate(mel.transpose(1,2))# save outputsf.write("location_relative_attention.wav",wav,sr)Train from ScatchClone the repo:git clone https://github.com/bshall/Tacotron
cd ./TacotronInstall requirements:pipenv installDownload and extract theLJ-Speech dataset:wget https://data.keithito.com/data/speech/LJSpeech-1.1.tar.bz2
tar -xvjf LJSpeech-1.1.tar.bz2Download the train splithereand extract it in the root directory of the repo.Extract Mel spectrograms and preprocess audio:pipenv run python preprocess.py path/to/LJSpeech-1.1 datasets/LJSpeech-1.1usage: preprocess.py [-h] in_dir out_dir

Preprocess an audio dataset.

positional arguments:
  in_dir      Path to the dataset directory
  out_dir     Path to the output directory

optional arguments:
  -h, --help  show this help message and exitTrain the model:pipenv run python train.py ljspeech datasets/LJSpeech-1.1 path/to/LJSpeech-1.1/metadata.csvusage: train.py [-h] [--resume RESUME] checkpoint_dir text_path dataset_dir

Train Tacotron with dynamic convolution attention.

positional arguments:
  checkpoint_dir   Path to the directory where model checkpoints will be saved
  text_path        Path to the dataset transcripts
  dataset_dir      Path to the preprocessed data directory

optional arguments:
  -h, --help       show this help message and exit
  --resume RESUME  Path to the checkpoint to resume fromPretrained ModelsPretrained weights for the LJSpeech model are availablehere.Notable Differences from the PaperTrained using a batch size of 64 on a single GPU (using automatic mixed precision).Used a gradient clipping threshold of 0.05 as it seems to stabilize the alignment with the smaller batch size.Used a different learning rate schedule (again to deal with smaller batch size).Used 80-bin (instead of 128 bin) log-Mel spectrograms.Acknowlegementshttps://github.com/keithito/tacotronhttps://github.com/PetrochukM/PyTorch-NLPhttps://github.com/fatchord/WaveRNN