Patent ID: 11915689
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 15:
16. The method of claim 15, wherein:
the quantization is residual vector quantization that encodes each embedding using a hierarchy of a plurality of vector quantizers that each generate a respective acoustic token from a corresponding vocabulary of acoustic tokens for the vector quantizer, wherein the hierarchy comprises one or more coarse vector quantizers at one or more first positions in the hierarchy and one or more fine vector quantizers at one or more last positions in the hierarchy, and
the set of one or more respective acoustic tokens at each second time step comprise, for each vector quantizer, a respective acoustic token selected from the vocabulary for the vector quantizer and being a prediction of a ground truth acoustic token that would be generated by the vector quantizer from a ground truth embedding generated by the encoder neural network at the second time step.