Package: tacotron-cli
tacotron-cliCommand-line interface (CLI) to train Tacotron 2 using .wav <=> .TextGrid pairs.Featurestrain phoneme stress separately (ARPAbet/IPA)train phoneme tone separately (IPA)train phoneme duration separately (IPA)train single/multi-speakertrain/synthesize on CPU or GPUsynthesis of paragraphscopy embeddings from one checkpoint to anothertrain using embeddings or one-hot encodingsInstallationpipinstalltacotron-cli--userUsageusage: tacotron-cli [-h] [-v] {create-mels,train,continue-train,validate,synthesize,synthesize-grids,analyze,add-missing-symbols} ...

Command-line interface (CLI) to train Tacotron 2 using .wav <=> .TextGrid pairs.

positional arguments:
  {create-mels,train,continue-train,validate,synthesize,synthesize-grids,analyze,add-missing-symbols}
                              description
    create-mels               create mel-spectrograms from audio files
    train                     start training
    continue-train            continue training from a checkpoint
    validate                  validate checkpoint(s)
    synthesize                synthesize lines from a file
    synthesize-grids          synthesize .TextGrid files
    analyze                   analyze checkpoint
    add-missing-symbols       copy missing symbols from one checkpoint to another

options:
  -h, --help                  show this help message and exit
  -v, --version               show program's version number and exitTrainingThe dataset structure need to follow the generic format ofspeech-dataset-parser, i.e., each TextGrid need to contain a tier in which all phonemes are separated into single intervals, e.g.,T|h|i|s| |i|s| |a| |t|e|x|t|..Tips:place stress directly to the vowel of the syllable, e.g.b|ˈo|d|iinstead ofˈb|o|d|i(body)place tone directly to the vowel of the syllable, e.g.ʈʂʰ|w|a˥˩|ninstead ofʈʂʰ|w|a|n˥˩(串)tone-characters which are considered:˥ ˦ ˧ ˨ ˩, e.g.,ɑ˥˩duration-characters which are considered:˘ ˑ ː, e.g.,ʌːnormalize the text, e.g., numbers should be written outsubstituted space by eitherSIL0,SIL1orSIL2depending on the duration of the pauseuseSIL0for no pauseuseSIL1for a short pause, for example after a comma...|v|i|ˈɛ|n|ʌ|,|SIL1|ˈɔ|s|t|ɹ|i|ʌ|...useSIL2for a longer pause, for example after a sentence:...|ˈɝ|θ|.|SIL2Note: only phonemes occurring in the TextGrids (on the selected tier) are possible to synthesizeSynthesisTo prepare a text for synthesis, following things need to be considered:each line in the text file will be synthesized as a single file, therefore it is recommended to place each sentence onto a single lineparagraphs can be separated by a blank lineeach symbol needs can be separated by an separator like|, e.g.s|ˌɪ|ɡ|ɝ|ˈɛ|tthis is useful if the model contains phonemes/symbols that consist of multiple characters, e.g.,ˈɛExample valid sentence: "As the overlying plate lifts up, it also forms mountain ranges." =>ˈæ|z|SIL0|ð|ʌ|SIL0|ˌoʊ|v|ɝ|l|ˈaɪ|ɪ|ŋ|SIL0|p|l|ˈeɪ|t|SIL0|l|ˈɪ|f|t|s|SIL0|ˈʌ|p|,|SIL1|ɪ|t|SIL0|ˈɔ|l|s|oʊ|SIL0|f|ˈɔ|ɹ|m|z|SIL0|m|ˈaʊ|n|t|ʌ|n|SIL0|ɹ|ˈeɪ|n|d͡ʒ|ʌ|z|.|SIL2Example invalid sentence: "Digestion is a vital process which involves the breakdown of food into smaller and smaller components, until they can be absorbed and assimilated into the body." =>daɪˈʤɛsʧʌn ɪz ʌ ˈvaɪtʌl ˈpɹɑˌsɛs wɪʧ ɪnˈvɑlvz ðʌ ˈbɹeɪkˌdaʊn ʌv fud ˈɪntu ˈsmɔlɝ ænd ˈsmɔlɝ kʌmˈpoʊnʌnts, ʌnˈtɪl ðeɪ kæn bi ʌbˈzɔɹbd ænd ʌˈsɪmʌˌleɪtɪd ˈɪntu ðʌ ˈbɑdi.Pretrained ModelsEnglishLJ Speech English TTSLJ Speech English TTS with explicit duration markersChineseTHCHS-30 Chinese TTSTHCHS-30 Chinese TTS with explicit duration markersAudio Example"The North Wind and the Sun were disputing which was the stronger, when a traveler came along wrapped in a warm cloak."Listen here(headphones recommended)Example SynthesisTo reproduce the audio example from above, you can use the following commands:# Create example directorymkdir~/example# Download pre-trained Tacotron model checkpointwgethttps://tuc.cloud/index.php/s/xxFCDMgEk8dZKbp/download/LJS-IPA-101500.pt-O~/example/checkpoint-tacotron.pt# Download pre-trained Waveglow model checkpointwgethttps://tuc.cloud/index.php/s/yBRaWz5oHrFwigf/download/LJS-v3-580000.pt-O~/example/checkpoint-waveglow.pt# Create text containing phonetic transcription of: "The North Wind and the Sun were disputing which was the stronger, when a traveler came along wrapped in a warm cloak."cat>~/example/text.txt<< EOFð|ʌ|SIL0|n|ˈɔ|ɹ|θ|SIL0|w|ˈɪ|n|d|SIL0|ˈæ|n|d|SIL0|ð|ʌ|SIL0|s|ˈʌ|n|SIL0|w|ɝ|SIL0|d|ɪ|s|p|j|ˈu|t|ɪ|ŋ|SIL0|h|w|ˈɪ|t͡ʃ|SIL0|w|ˈɑ|z|SIL0|ð|ʌ|SIL0|s|t|ɹ|ˈɔ|ŋ|ɝ|,|SIL1|h|w|ˈɛ|n|SIL0|ʌ|SIL0|t|ɹ|ˈæ|v|ʌ|l|ɝ|SIL0|k|ˈeɪ|m|SIL0|ʌ|l|ˈɔ|ŋ|SIL0|ɹ|ˈæ|p|t|SIL0|ɪ|n|SIL0|ʌ|SIL0|w|ˈɔ|ɹ|m|SIL0|k|l|ˈoʊ|k|.|SIL2EOF# Synthesize text to mel-spectrogramtacotron-clisynthesize\~/example/checkpoint-tacotron.pt\~/example/text.txt\--sep"|"# Install waveglow-cli for synthesis of mel-spectrogramspipinstallwaveglow-cli--user# Synthesize mel-spectrogram to wavwaveglow-clisynthesize\~/example/checkpoint-waveglow.pt\~/example/text-o# Resulting wav is written to: ~/example/text/1-1.npy.wavRoadmapOutsource method to convert audio files to mel-spectrograms before trainingBetter loggingProvide more pre-trained modelsAdding testsDevelopment setup# updatesudoaptupdate# install Python 3.8-3.11 for ensuring that tests can be runsudoaptinstallpython3-pip\python3.8python3.8-devpython3.8-distutilspython3.8-venv\python3.9python3.9-devpython3.9-distutilspython3.9-venv\python3.10python3.10-devpython3.10-distutilspython3.10-venv\python3.11python3.11-devpython3.11-distutilspython3.11-venv# install pipenv for creation of virtual environmentspython3.8-mpipinstallpipenv--user# check out repogitclonehttps://github.com/stefantaubert/tacotron.gitcdtacotron# create virtual environmentpython3.8-mpipenvinstall--devRunning the tests# first install the tool like in "Development setup"# then, navigate into the directory of the repo (if not already done)cdtacotron# activate environmentpython3.8-mpipenvshell# run teststoxFinal lines of test result output:py38: commands succeeded
py39: commands succeeded
py310: commands succeeded
py311: commands succeeded
congratulations :)LicenseMIT LicenseAcknowledgmentsModel code adapted fromNvidia.Papers:Tacotron: Towards End-to-End Speech SynthesisNatural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram PredictionsFunded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 416228727 – CRC 1410CitationIf you want to cite this repo, you can use the BibTeX-entry generated by GitHub (seeAbout => Cite this repository).Taubert, S. (2024). tacotron-cli (Version 0.0.5) [Computer software]. [https://doi.org/10.5281/zenodo.10568731](https://doi.org/10.5281/zenodo.10568731)Cited byTaubert, S., Sternkopf, J., Kahl, S., & Eibl, M. (2022). A Comparison of Text Selection Algorithms for Sequence-to-Sequence Neural TTS. 2022 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), 1–6.https://doi.org/10.1109/ICSPCC55723.2022.9984283Albrecht, S., Tamboli, R., Taubert, S., Eibl, M., Rey, G. D., & Schmied, J. (2022). Towards a Vowel Formant Based Quality Metric for Text-to-Speech Systems: Measuring Monophthong Naturalness. 2022 IEEE 9th International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), 1–6.https://doi.org/10.1109/CIVEMSA53371.2022.9853712