Patent ID: 11881210
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations comprising:
receiving a text utterance having one or more words, each word having at least one syllable, each syllable having at least one phoneme;
splitting, by the data processing hardware, using a tokenizer, the text utterance into a sequence of wordpiece units;
generating, using a Bidirectional Encoder Representations from Transformers (BERT) model, a sequence of wordpiece embeddings, each wordpiece embedding corresponding to one of the word piece units that are associated with one of the one or more words of the text utterance;
selecting an utterance embedding for the text utterance, the utterance embedding representing an intended prosody;
for each syllable, using the selected utterance embedding and a prosody model that incorporates the BERT model:
when the word that includes the syllable includes a plurality of wordpiece units from the sequence of wordpiece units, selecting the single wordpiece embedding from the sequence of wordpiece embeddings that corresponds to the initial wordpiece unit of the plurality of wordpiece units from the sequence of wordpiece units to represent the word that includes the syllable;
generating a corresponding prosodic syllable embedding for the syllable based on only the selected single wordpiece embedding corresponding to the initial wordpiece unit of the plurality of wordpiece units from the sequence of wordpiece units that includes the syllable that is selected to represent the word that includes the syllable; and
predicting a duration of the syllable by encoding linguistic features of each phoneme of the syllable with the corresponding prosodic syllable embedding for the syllable; and

generating, using the prosody model, a prosodic representation for the text utterance based on the predicted durations of the syllables.