Patent ID: 11881210
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 20:
21. The system of claim 20, wherein encoding each reference audio signal into a corresponding fixed-length utterance embedding comprises:
generating, using the BERT model, a sequence of wordpiece embeddings from the transcript of the corresponding reference audio signal;
sampling, from the corresponding reference audio signal, a sequence of fixed-length reference frames providing a duration, pitch contour, and/or energy contour that represents the corresponding prosody of the reference audio signal; and
for each syllable in the reference audio signal:
encoding phone-level linguistic features associated with each phoneme in the syllable into a phone feature-based syllable embedding;
encoding the fixed-length reference frames associated with the syllable into a frame-based syllable embedding, the frame-based syllable embedding indicative of a duration, pitch, and/or energy associated with the corresponding syllable; and
encoding, into a corresponding prosodic syllable embedding for the syllable, the phoneme feature-based and frame-based syllable embeddings with syllable-level linguistic features associated with the syllable, sentence-level linguistic features associated with the reference audio signal, and a wordpiece embedding from the sequence of wordpiece embeddings generated by the BERT model that is associated with a word that includes the corresponding syllable.