Patent ID: 11961515
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 23:
24. The computer-implemented method of claim 13, wherein the speech recognition model comprises a Transformer-Transducer (T-T) model and the operations further comprise:
receiving, as input to the audio encoder of the T-T model, a plurality of unlabeled audio samples corresponding to spoken utterances not paired with corresponding transcriptions;
generating, by the audio encoder, at each of a plurality of time steps, a sequence of acoustic frames extracted from audio data characterizing a spoken utterance;
receiving, as input to a label encoder of the T-T model, a sequence of non-blank symbols output by a final softmax layer;
generating, by the label encoder, at each of the plurality of time steps, a dense representation;
receiving, as input to a joint network of the T-T model, the higher order feature representation generated by the audio encoder at each of the plurality of time steps and the dense representation generated by the label encoder at each of the plurality of time steps; and
generating, by the joint network, at each of the plurality of time steps, a probability distribution over possible speech recognition hypothesis at the corresponding time step,
wherein the audio encoder comprises a neural network having a stack of strided convolutional layers and transformer layers.