Patent ID: 11960852
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 9:
10. A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising:
receiving, as input to a direct speech-to-speech translation (S2ST) model, an input speech representation corresponding to an utterance spoken by a source speaker in a first language and in a first voice;
encoding, by an encoder of the S2ST model, the input speech representation into a hidden feature representation;
generating, by an attention module of the S2ST model, a context vector that attends to the hidden feature representation encoded by the encoder;
receiving, at a decoder of the S2ST model, the context vector generated by the attention module;
predicting, by the decoder, a phoneme representation corresponding to a translation of the utterance in a second different language, the phoneme representation comprising a sequence of phonemes;
receiving, at a synthesizer of the S2ST model, the context vector and the phoneme representation; and
generating, by the synthesizer, a translated synthesized speech representation corresponding to the translation of the utterance spoken in the different second language and in the first voice,
wherein the S2ST model is trained end-to-end on pairs of parallel source language and target language utterances each in a same voice, the S2ST model trained to not translate input speech representations corresponding to the source language utterances into corresponding translated synthesized speech representations that are in a different voice than the corresponding source language utterances.