Package: gruut
GruutA tokenizer, text cleaner, andIPAphonemizer for several human languages that supportsSSML.fromgruutimportsentencestext='He wound it around the wound, saying "I read it was $10 to read."'forsentinsentences(text,lang="en-us"):forwordinsent:ifword.phonemes:print(word.text,*word.phonemes)which outputs:He h ˈi
wound w ˈaʊ n d
it ˈɪ t
around ɚ ˈaʊ n d
the ð ə
wound w ˈu n d
, |
saying s ˈeɪ ɪ ŋ
I ˈaɪ
read ɹ ˈɛ d
it ˈɪ t
was w ə z
ten t ˈɛ n
dollars d ˈɑ l ɚ z
to t ə
read ɹ ˈi d
. ‖Note that "wound" and "read" have different pronunciations when used in different (grammatical) contexts.Asubset of SSMLis also supported:fromgruutimportsentencesssml_text="""<?xml version="1.0" encoding="ISO-8859-1"?><speak version="1.1" xmlns="http://www.w3.org/2001/10/synthesis"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.w3.org/2001/10/synthesishttp://www.w3.org/TR/speech-synthesis11/synthesis.xsd"xml:lang="en-US">
Today at 4pm, 2/1/2000.
<s xml:lang="it">Un mese fà, 2/1/2000.
</speak>"""forsentinsentences(ssml_text,ssml=True):forwordinsent:ifword.phonemes:print(sent.idx,word.lang,word.text,*word.phonemes)with the output:0 en-US Today t ə d ˈeɪ
0 en-US at ˈæ t
0 en-US four f ˈɔ ɹ
0 en-US P p ˈi
0 en-US M ˈɛ m
0 en-US , |
0 en-US February f ˈɛ b j u ˌɛ ɹ i
0 en-US first f ˈɚ s t
0 en-US , |
0 en-US two t ˈu
0 en-US thousand θ ˈaʊ z ə n d
0 en-US . ‖
1 it Un u n
1 it mese ˈm e s e
1 it fà f a
1 it , |
1 it due d j u
1 it gennaio d͡ʒ e n n ˈa j o
1 it duemila d u e ˈm i l a
1 it . ‖Seethe documentationfor more details.InstallationpipinstallgruutLanguages besides English can be added during installation. For example, with French and Italian support:pipinstall-f'https://synesthesiam.github.io/prebuilt-apps/'gruut[fr,it]The extra pip repo is needed for an updatednum2words forkthat includes support for more languages.You may alsomanually download language filesand use put them in$XDG_CONFIG_HOME/gruut/($HOME/.config/gruutby default).gruut will look for language files in the directory$XDG_CONFIG_HOME/gruut/<lang>/if the corresponding Python package is not installed. Note that<lang>here is thefulllanguage name, e.g.de-deinstead of justde.Supported Languagesgruut currently supports:Arabic (ar)Czech (csorcs-cz)German (deorde-de)English (enoren-us)Spanish (esores-es)Farsi/Persian (fa)French (frorfr-fr)Italian (itorit-it)Luxembourgish (lb)Dutch (nl)Russian (ruorru-ru)Swedish (svorsv-se)Swahili (sw)The goal is to support all ofvoice2json's languagesDependenciesPython 3.7 or higherLinuxTested on Debian Bullseyenum2words forkandBabelCurrency/number handlingnum2words fork includes additional language support (Arabic, Farsi, Swedish, Swahili)gruut-ipaIPApronunciation manipulationpycrfsuitePart of speech tagging and grapheme to phoneme modelspydateparserDate parsing for multiple languagesNumbers, Dates, and Moregruutcan automatically verbalize numbers, dates, and other expressions. This is done in a locale-aware manner for both parsing and verbalization, so "1/1/2020" may be interpreted as "M/D/Y" or "D/M/Y" depending on the word or sentence's language (e.g.,<s lang="...">).The following types of expressions can be automatically expanded into words bygruut:Numbers - "123" to "one hundred and twenty three" (disable withverbalize_numbers=Falseor--no-numbers)Relies onBabelfor parsing andnum2wordsfor verbalizationDates - "1/1/2020" to "January first, twenty twenty" (disable withverbalize_dates=Falseor--no-dates)Relies onpydateparserfor parsing and bothBabelandnum2wordsfor verbalizationCurrency - "$10" to "ten dollars" (disable withverbalize_currency=Falseor--no-currency)Relies onBabelfor parsing and bothBabelandnum2wordsfor verbalizationTimes - "12:01am" to "twelve oh one A M" (disable withverbalize_times=Falseor--no-times)English onlyRelies onnum2wordsfor verbalizationCommand-Line UsageThegruutmodule can be executed withpython3 -m gruut --language <LANGUAGE> <TEXT>or with thegruutcommand (fromsetup.py).Thegruutcommand is line-oriented, consuming text and producingJSONL.
You will probably want to installjqto manipulate theJSONLoutput fromgruut.Plain TextTakes raw text and outputsJSONLwith cleaned words/tokens.echo'This, right here, is some "RAW" text!'\|gruut--languageen-us\|jq--raw-output'.words[].text'This
,
right
here
,
is
some"RAW"text
!More information is available in the full JSON output:gruut--languageen-us'More  text.'|jq.Output:{"idx":0,"text":"More text.","text_with_ws":"More text.","text_spoken":"More text","par_idx":0,"lang":"en-us","voice":"","words":[{"idx":0,"text":"More","text_with_ws":"More ","leading_ws":"","training_ws":" ","sent_idx":0,"par_idx":0,"lang":"en-us","voice":"","pos":"JJR","phonemes":["m","ˈɔ","ɹ"],"is_major_break":false,"is_minor_break":false,"is_punctuation":false,"is_break":false,"is_spoken":true,"pause_before_ms":0,"pause_after_ms":0},{"idx":1,"text":"text","text_with_ws":"text","leading_ws":"","training_ws":"","sent_idx":0,"par_idx":0,"lang":"en-us","voice":"","pos":"NN","phonemes":["t","ˈɛ","k","s","t"],"is_major_break":false,"is_minor_break":false,"is_punctuation":false,"is_break":false,"is_spoken":true,"pause_before_ms":0,"pause_after_ms":0},{"idx":2,"text":".","text_with_ws":".","leading_ws":"","training_ws":"","sent_idx":0,"par_idx":0,"lang":"en-us","voice":"","pos":null,"phonemes":["‖"],"is_major_break":true,"is_minor_break":false,"is_punctuation":false,"is_break":true,"is_spoken":false,"pause_before_ms":0,"pause_after_ms":0}],"pause_before_ms":0,"pause_after_ms":0}For the whole input line and each word, thetextproperty contains the processed input text with normalized whitespace whiletext_with_wsretains the original whitespace. Thetext_spokenproperty only contains words that are spoken, so punctuation and breaks are excluded.Within each word, there is:idx- zero-based index of the word in the sentencesent_idx- zero-based index of the sentence in the input textpos- part of speech tag (if available)phonemes- list ofIPAphonemes for the word (if available)is_minor_break-trueif "word" separates phrases (comma, semicolon, etc.)is_major_break-trueif "word" separates sentences (period, question mark, etc.)is_break-trueif "word" is a major or minor breakis_punctuation-trueif "word" is a surrounding punctuation mark (quote, bracket, etc.)is_spoken-trueif not a break or punctuationSeepython3 -m gruut <LANGUAGE> --helpfor more options.SSMLA subset ofSSMLis supported:<speak>- wrap around SSML textlang- set language for document<p>- paragraphlang- set language for paragraph
- sentence (disables automatic sentence breaking)lang- set language for sentence<w>/<token>- word (disables automatic tokenization)lang- set language for wordrole- set word role (seeword roles)<lang lang="...">- set language inner text<voice name="...">- set voice of inner text<say-as interpret-as="">- force interpretation of inner textinterpret-asone of "spell-out", "date", "number", "time", or "currency"format- way to format text depending oninterpret-asnumber - one of "cardinal", "ordinal", "digits", "year"date - string with "d" (cardinal day), "o" (ordinal day), "m" (month), or "y" (year)<break time="">- Pause for given amount of timetime - seconds ("123s") or milliseconds ("123ms")<mark name="">- User-defined mark (marks_beforeandmarks_afterattributes of words/sentences)name - name of mark<sub alias="">- substitutealiasfor inner text<phoneme ph="...">- supply phonemes for inner textph- phonemes for each word of inner text, separated by whitespace<lexicon id="...">- inline or external pronunciation lexiconid- unique id of lexicon (used in<lookup ref="...">)uri- if empty or missing, lexicon is inlineOne or more<lexeme>child elements with:Optionalrole="..."([word roles][#word-roles] separated by whitespace)<grapheme>WORD</grapheme>- word text<phoneme>P H O N E M E S</phoneme>- word pronunciation (phonemes separated by whitespace)<lookup ref="...">- use pronunciation lexicon for child elementsref- id from a<lexicon id="...">Word RolesDuring phonemization, word roles are used to disambiguate pronunciations. Unless manually specified, a word's role is derived from its part of speech tag asgruut:<TAG>. For initialisms andspell-out, the rolegruut:letteris used to indicate that e.g., "a" should be spoken as/eɪ/instead of/ə/.Foren-us, the following additional roles are available from the part-of-speech tagger:gruut:CD- numbergruut:DT- determinergruut:IN- preposition or subordinating conjunctiongruut:JJ- adjectivegruut:NN- noungruut:PRP- personal pronoungruut:RB- adverbgruut:VB- verbgruut:VB- verb (past tense)Inline LexiconsInlinepronunciation lexiconsare supported via the<lexicon>and<lookup>tags. gruut diverges slightly from theSSML standardhere by allowing lexicons to be defined within the SSML document itself (urlis blank or missing). Additionally, theidattribute of the<lexicon>element can be left off to indicate a "default" inline lexicon that does not require a corresponding<lookup>tag.For example, the following document will yield three different pronunciations for the word "tomato":<?xml version="1.0"?><speakversion="1.1"xmlns="http://www.w3.org/2001/10/synthesis"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.w3.org/2001/10/synthesishttp://www.w3.org/TR/speech-synthesis11/synthesis.xsd"xml:lang="en-US"><lexiconxml:id="test"alphabet="ipa"><lexeme><grapheme>tomato</grapheme><phoneme><!-- Individual phonemes are separated by whitespace -->təmˈɑtoʊ</phoneme></lexeme><lexeme><graphemerole="fake-role">tomato</grapheme><phoneme><!-- Made up pronunciation for fake word role -->təmˈitoʊ</phoneme></lexeme></lexicon><w>tomato</w><lookupref="test"><w>tomato</w><wrole="fake-role">tomato</w></lookup></speak>The first "tomato" will be looked up in the U.S. English lexicon (/t ə m ˈeɪ t oʊ/). Within the<lookup>tag's scope, the second and third "tomato" words will be looked up in the inline lexicon. The third "tomato" word has aroleattached  (selecting a made up pronunciation in this case).Even further from the SSML standard, gruut allows you to leave off the<lexicon>id entirely. With noid, a<lookup>tag is no longer needed, allowing you to override the pronunciation of any word in the document:<?xml version="1.0"?><speakversion="1.1"xmlns="http://www.w3.org/2001/10/synthesis"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.w3.org/2001/10/synthesishttp://www.w3.org/TR/speech-synthesis11/synthesis.xsd"xml:lang="en-US"><!-- No id means change all words without a lookup --><lexicon><lexeme><grapheme>tomato</grapheme><phoneme>təmˈɑtoʊ</phoneme></lexeme></lexicon><w>tomato</w></speak>This will yield a pronunciation of/t ə m ˈɑ t oʊ/for all instances of "tomato" in the document (unless they have a<lookup>).Intended Audiencegruut is useful for transforming raw text into phonetic pronunciations, similar tophonemizer. Unlike phonemizer, gruut looks up words in a pre-built lexicon (pronunciation dictionary) or guesses word pronunciations with a pre-trained grapheme-to-phoneme model. Phonemes for each language come from acarefully chosen inventory.For each supported language, gruut includes a:A word pronunciation lexicon built from open source dataSeepron_dictA pre-trained grapheme-to-phoneme model for guessing word pronunciationsSome languages also include:A pre-trained part of speech tagger built from open source data:Seeuniversal dependencies