Package: larynx
Larynx🎥DEMO VIDEOOffline end-to-end text to speech system usinggruutandonnx(architecture). There are50 voices available across 9 languages.curlhttps://raw.githubusercontent.com/rhasspy/larynx/master/docker/larynx-server\>~/bin/larynx-server&&chmod+755~/bin/larynx-server
larynx-serverVisithttp://localhost:5002for the test page. Seehttp://localhost:5002/openapi/for HTTP endpoint documentation.Supports asubset of SSMLthat can use multiple voices and languages!<speak>The1stthingtorememberisthat9languagesaresupportedinLarynxTTSasof10/19/2021at10:39am.<voicename="harvard">
Thecurrentvoicecanbechanged!
</voice><voicename="northern_english_male">
Breaksarepossible
<breaktime="0.5s"/>
betweensentences.
</voice><slang="en">Onelanguageisneverenough
<slang="de">EineSpracheistniemalsgenug
<slang="sw">Lughamojahaitoshi
</speak>Larynx's goals are:"Good enough" synthesis to avoid using a cloud serviceFaster than realtime performance on a Raspberry Pi 4 (with low quality vocoder)Broad language support (9 languages)Voices trained purely from public datasetsYou can use Larynx to:Host atext to speech HTTP endpointSynthesize texton the command-lineRead a bookto youSamplesListen to voice samplesfrom all of thepre-trained voices.Docker InstallationPre-built Docker images are available for the following platforms:linux/amd64- desktop/laptop/serverlinux/arm64- Raspberry Pi 64-bitlinux/arm/v7- Raspberry Pi 32-bitThese images include a single English voice, butmany more can be downloadedfrom within the web interface.Thelarynxandlarynx-servershell scripts wrap the Docker images, allowing you to use Larynx as a command-line tool.To manually run the Larynx web server in Docker:dockerrun\-it\-p5002:5002\-e"HOME=${HOME}"\-v"$HOME:${HOME}"\-v/usr/share/ca-certificates:/usr/share/ca-certificates\-v/etc/ssl/certs:/etc/ssl/certs\-w"${PWD}"\--user"$(id-u):$(id-g)"\rhasspy/larynxDownloaded voices will be stored in${HOME}/.local/share/larynx.Visithttp://localhost:5002for the test page. Seehttp://localhost:5002/openapi/for HTTP endpoint documentation.Debian InstallationPre-built Debian packages forbullseyeareavailable for downloadwith the namelarynx-tts_<VERSION>_<ARCH>.debwhereARCHis one ofamd64(most desktops, laptops),armhf(32-bit Raspberry Pi), andarm64(64-bit Raspberry Pi)Example installation on a typical desktop:sudoaptinstall./larynx-tts_<VERSION>_amd64.debFrom there, you may run thelarynxcommand orlarynx-serverto start the web server (http://localhost:5002).Python InstallationYou may need to install the following dependencies (besides Python 3.7+):sudoapt-getinstalllibopenblas-baselibgomp1libatomic1On 32-bit ARM systems (Raspberry Pi), you will also need:sudoapt-getinstalllibatlas3-baselibgfortran5Next, create a Python virtual environment:python3-mvenvlarynx_venvsourcelarynx_venv/bin/activate

pip3install--upgradepip
pip3install--upgradewheelsetuptoolsNext, install larynx:pip3install-f'https://synesthesiam.github.io/prebuilt-apps/'-f'https://download.pytorch.org/whl/cpu/torch_stable.html'larynxThen runlarynxorlarynx.serverfor the web server. You may also execute the Python module directly withpython3 -m larynxandpython3 -m larynx.server.Voice/Vocoder DownloadVoices and vocoders are automatically downloaded when used on the command-line or in the web server. You can alsomanually download each voice. Extract them to${HOME}/.local/share/larynx/voicesso that the directory structure follows the pattern${HOME}/.local/share/larynx/voices/<language>,<voice>.Command-Line InterfaceLarynx has a flexible command-line interface, available with:Thelarynx scriptfor DockerThelarynxcommand from the Debian packagelarynxorpython3 -m larynxfor Python installationsBasic Synthesislarynx-v<VOICE>"<TEXT>">output.wavwhere<VOICE>is a language name (en,de, etc) or a voice name (ljspeech,thorsten, etc).<TEXT>may contain multiple sentences, which will be combined in the final output WAV file. These can also besplit into separate WAV files.To adjust the quality of the output, use-q <QUALITY>where<QUALITY>is "high" (slowest), "medium", or "low" (fastest).SSML Synthesislarynx--ssml-v<VOICE>"<SSML>">output.wavwhere<SSML>is validSSML. Not all features are supported; for example:Breaks (pauses) can only occur between sentences and can only be specified in seconds or millisecondsVoices can only be referenced by nameCustom lexicons are not yet supported (you can use<phoneme ph="...">, however)If your SSML contains<mark>tags, add--mark-file <FILE>to the command-line. As the marks are encountered (between sentences), their names will be written on separate lines to the file.CUDA Accelerated SynthesisThe--cudaflag will make use of a GPU if its available to PyTorch:larynx--cuda'This is spoken on the GPU.'>output.wavAdding the--halfflag will enable half-precision inference, which is often faster:larynx--cuda--half'This is spoken on the GPU even faster.'>output.wavFor CUDA acceleration to work, your voice must contain a PyTorch checkpoint file (generator.pth). Older Larynx voices did not have these, so you may need tore-download your voices.Long TextsIf your text is very long, and you would like to listen to it as its being synthesized, use the--raw-streamoption:larynx-ven--raw-stream<long.txt|aplay-r22050-c1-fS16_LEEach input line will be synthesized and written the standard out as raw 16-bit 22050Hz mono PCM. By default, 5 sentences will be kept in an output queue, only blocking synthesis when the queue is full. You can adjust this value with--raw-stream-queue-size. Additionally, you can adjust--max-thread-workersto change how many threads are available for synthesis.If your long text is fixed-width with blank lines separating paragraphs like those fromProject Gutenberg, use the--process-on-blank-lineoption so that sentences will not be broken at line boundaries. For example, you can listen to "Alice in Wonderland" like this:curl--output-'https://www.gutenberg.org/files/11/11-0.txt'|\larynx-vek--raw-stream--process-on-blank-line|aplay-r22050-c1-fS16_LEMultiple WAV OutputWith--output-dirset to a directory, Larynx will output a separate WAV file for each sentence:larynx-ven'Test 1. Test 2.'--output-dir/path/to/wavsBy default, each WAV file will be named using the (slightly modified) text of the sentence. You can have WAV files named using a timestamp instead with--output-naming time. For full control of the output naming, the--csvcommand-line flag indicates that each sentence is of the formid|textwhereidwill be the name of the WAV file.cat<< EOF |s01|The birch canoe slid on the smooth planks.s02|Glue the sheet to the dark blue background.s03|It's easy to tell the depth of a well.s04|These days a chicken leg is a rare dish.s05|Rice is often served in round bowls.s06|The juice of lemons makes fine punch.s07|The box was thrown beside the parked truck.s08|The hogs were fed chopped corn and garbage.s09|Four hours of steady work faced us.s10|Large size in stockings is hard to sell.EOFlarynx--csv--voiceen--output-dir/path/to/wavsInteractive ModeWith no text input and no output directory, Larynx will switch into interactive mode. After entering a sentence, it will be played with--play-command(default isplayfrom SoX).larynx-ven
Readingtextfromstdin...
Helloworld!<ENTER>UseCTRL+DorCTRL+Cto exit.GlowTTS SettingsThe GlowTTS voices support two additional parameters:--noise-scale- determines the speaker volatility during synthesis (0-1, default is  0.667)--length-scale- makes the voice speaker slower (> 1) or faster (< 1)Vocoder Settings--denoiser-strength- runs the denoiser if > 0; a small value like 0.005 is a good place to start.List Voices and Vocoderslarynx--listMaryTTS Compatible APITo use Larynx as a drop-in replacement for aMaryTTSserver (e.g., for use withHome Assistant), run:dockerrun\-it\-p59125:5002\-e"HOME=${HOME}"\-v"$HOME:${HOME}"\-v/usr/share/ca-certificates:/usr/share/ca-certificates\-v/etc/ssl/certs:/etc/ssl/certs\-w"${PWD}"\--user"$(id-u):$(id-g)"\rhasspy/larynxThe/processHTTP endpoint should now work for voices formatted as<LANG>or<VOICE>, e.g.enorharvard.You can specify the vocoder quality by adding;<QUALITY>to the MaryTTS voice whereQUALITYis "high", "medium", or "low".For example:en;lowwill use the lowest quality (but fastest) vocoder. This is usually necessary to get decent performance on a Raspberry Pi.SSMLA subset ofSSMLis supported (use--ssml):<speak>- wrap around SSML textlang- set language for document
- sentence (disables automatic sentence breaking)lang- set language for sentence<w>/<token>- word (disables automatic tokenization)<voice name="...">- set voice of inner textvoice- name or language of voice<say-as interpret-as="">- force interpretation of inner textinterpret-asone of "spell-out", "date", "number", "time", or "currency"format- way to format text depending oninterpret-asnumber - one of "cardinal", "ordinal", "digits", "year"date - string with "d" (cardinal day), "o" (ordinal day), "m" (month), or "y" (year)<break time="">- Pause for given amount of timetime - seconds ("123s") or milliseconds ("123ms")<mark name="">- User-defined mark (written to--mark-fileor part ofTextToSpeechResult)name - name of mark<sub alias="">- substitutealiasfor inner text<phoneme ph="...">- supply phonemes for inner textph- phonemes for each word of inner text, separated by whitespace<lexicon id="...">- inline pronunciation lexiconid- unique id of lexicon (used in<lookup ref="...">)One or more<lexeme>child elements with:<grapheme role="...">WORD</grapheme>- word text (optional [role][#word-roles])<phoneme>P H O N E M E S</phoneme>- word pronunciation (phonemes separated by whitespace)<lookup ref="...">- use inline pronunciation lexicon for child elementsref- id from a<lexicon id="...">Word RolesDuring phonemization, word roles are used to disambiguate pronunciations. Unless manually specified, a word's role is derived from its part of speech tag asgruut:<TAG>. For initialisms andspell-out, the rolegruut:letteris used to indicate that e.g., "a" should be spoken as/eɪ/instead of/ə/.Foren-us, the following additional roles are available from the part-of-speech tagger:gruut:CD- numbergruut:DT- determinergruut:IN- preposition or subordinating conjunctiongruut:JJ- adjectivegruut:NN- noungruut:PRP- personal pronoungruut:RB- adverbgruut:VB- verbgruut:VB- verb (past tense)Inline LexiconsInlinepronunciation lexiconsare supported via the<lexicon>and<lookup>tags. gruut diverges slightly from theSSML standardhere by only allowing lexicons to be defined within the SSML document itself. Additionally, theidattribute of the<lexicon>element can be left off to indicate a "default" inline lexicon that does not require a corresponding<lookup>tag.For example, the following document will yield three different pronunciations for the word "tomato":<?xml version="1.0"?><speakversion="1.1"xmlns="http://www.w3.org/2001/10/synthesis"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.w3.org/2001/10/synthesishttp://www.w3.org/TR/speech-synthesis11/synthesis.xsd"xml:lang="en-US"><lexiconxml:id="test"alphabet="ipa"><lexeme><grapheme>tomato</grapheme><phoneme><!-- Individual phonemes are separated by whitespace -->təmˈɑtoʊ</phoneme></lexeme><lexeme><graphemerole="fake-role">tomato</grapheme><phoneme><!-- Made up pronunciation for fake word role -->təmˈitoʊ</phoneme></lexeme></lexicon><w>tomato</w><lookupref="test"><w>tomato</w><wrole="fake-role">tomato</w></lookup></speak>The first "tomato" will be looked up in the U.S. English lexicon (/t ə m ˈeɪ t oʊ/). Within the<lookup>tag's scope, the second and third "tomato" words will be looked up in the inline lexicon. The third "tomato" word has aroleattached  (selecting a made up pronunciation in this case).Even further from the SSML standard, gruut allows you to leave off the<lexicon>id entirely. With noid, a<lookup>tag is no longer needed, allowing you to override the pronunciation of any word in the document:<?xml version="1.0"?><speakversion="1.1"xmlns="http://www.w3.org/2001/10/synthesis"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.w3.org/2001/10/synthesishttp://www.w3.org/TR/speech-synthesis11/synthesis.xsd"xml:lang="en-US"><!-- No id means change all words without a lookup --><lexicon><lexeme><grapheme>tomato</grapheme><phoneme>təmˈɑtoʊ</phoneme></lexeme></lexicon><w>tomato</w></speak>This will yield a pronunciation of/t ə m ˈɑ t oʊ/for all instances of "tomato" in the document (unless they have a<lookup>).Text to Speech ModelsGlowTTS(50 voices)English (en-us, 27 voices)blizzard_fls (F, accent,Blizzard)blizzard_lessac (F,Blizzard)cmu_aew (M,Arctic)cmu_ahw (M,Arctic)cmu_aup (M, accent,Arctic)cmu_bdl (M,Arctic)cmu_clb (F,Arctic)cmu_eey (F,Arctic)cmu_fem (M,Arctic)cmu_jmk (M,Arctic)cmu_ksp (M, accent,Arctic)cmu_ljm (F,Arctic)cmu_lnh (F,Arctic)cmu_rms (M,Arctic)cmu_rxr (M,Arctic)cmu_slp (F, accent,Arctic)cmu_slt (F,Arctic)ek (F, accent,M-AILabs)harvard (F, accent,CC/Attr/NC)kathleen (F,CC0)ljspeech (F,Public Domain)mary_ann (F,M-AILabs)northern_english_male (M,CC/Attr/SA)scottish_english_male (M,CC/Attr/SA)southern_english_female (F,CC/Attr/SA)southern_english_male (M,CC/Attr/SA)judy_bieber (F,M-AILabs)German (de-de, 7 voices)eva_k (F,M-AILabs)hokuspokus (F,CC0)karlsson (M,M-AILabs)kerstin (F,CC0)pavoque (M,CC4/BY/NC/SA)rebecca_braunert_plunkett (F,M-AILabs)thorsten (M,CC0)French (fr-fr, 3 voices)gilles_le_blanc (M,M-AILabs)siwis (F,CC/Attr)tom (M,ODbL)Spanish (es-es, 2 voices)carlfm (M, public domain)karen_savage (F,M-AILabs)Dutch (nl, 4 voices)bart_de_leeuw (M,Apache2)flemishguy (M,CC0)rdh (M,CC0)nathalie (F,CC0)Italian (it-it, 2 voices)lisa (F,M-AILabs)riccardo_fasol (M,Apache2)Swedish (sv-se, 1 voice)talesyntese (M,CC0)Swahili (sw, 1 voice)blblia_takatifu (M,Sermon Online)Russian (ru-ru, 3 voices)hajdurova (F,M-AILabs)nikolaev (M,M-AILabs)minaev (M,M-AILabs)VocodersHi-Fi GANUniversal large (slowest)VCTK "small"VCTK "medium" (fastest)BenchmarksThe following benchmarks were run on:Core i7-8750H (amd64)Raspberry Pi 4 (aarch64)Raspberry Pi 3 (armv7l)Multiple runs were done at each quality level, with the first run being discarded so that cache for the model files was hot.The RTF (real-time factor) is computed as the time taken to synthesize audio divided by the duration of the synthesized audio. An RTF less than 1 indicates that audio was able to be synthesized faster than real-time.PlatformQualityRTFamd64high0.25amd64medium0.06amd64low0.05------------------aarch64high4.28aarch64medium1.82aarch64low0.56------------------armv7lhigh16.83armv7lmedium7.16armv7llow2.22See the benchmarking scripts inscripts/for more details.ArchitectureLarynx breaks text to speech into 4 distinct steps:Text toIPAphonemes (gruut)Phonemes to ids (phonemes.txtfile from voice)Phoneme ids to mel spectrograms (glow-tts)Mel spectrograms to waveforms (hifi-gan)Voices are trained on phoneme ids and mel spectrograms. For each language, the voice with the most data available was used as a base model and fine-tuned.