Patent ID: 11915689
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 29:
30. One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising a method for generating a prediction of an audio signal, the method comprising:
receiving a request to generate an audio signal having a respective audio sample at each of a plurality of output time steps spanning a time window conditioned on an input;
processing the input using an embedding neural network to map the input to one or more embedding tokens;
generating a semantic representation of the audio signal that specifies a respective semantic token at each of a plurality of first time steps spanning the time window, each semantic token being selected from a vocabulary of semantic tokens conditioned on the embedding tokens and representing semantic content of the audio signal at the corresponding first time step;
generating, using one or more generative neural networks and conditioned on at least the semantic representation and the embedding tokens, an acoustic representation of the audio signal, the acoustic representation specifying a set of one or more respective acoustic tokens at each of a plurality of second time steps spanning the time window, the one or more respective acoustic tokens at each second time step representing acoustic properties of the audio signal at the corresponding second time step; and
processing at least the acoustic representation using a decoder neural network to generate the prediction of the audio signal.