Patent ID: 11961515
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising:
receiving a plurality of unlabeled audio samples corresponding to spoken utterances not paired with corresponding transcriptions:
at a target branch of a contrastive Siamese network:
generating, using an audio encoder of a speech recognition model, a sequence of encoder outputs for the plurality of unlabeled audio samples; and
modifying time characteristics of the encoder outputs to generate a sequence of target branch outputs;

at an augmentation branch of the contrastive Siamese network:
performing augmentation on the unlabeled audio samples;
generating, using the audio encoder of the speech recognition model, a sequence of augmented encoder outputs for the augmented unlabeled audio samples; and
generating, using a prediction network configured to receive the sequence of augmented encoder outputs, predictions of the sequence of target branch outputs generated at the target branch;

determining an unsupervised loss term based on the target branch outputs generated at the target branch and the predictions of the sequence of target branch outputs generated at the augmentation branch; and
updating parameters of the audio encoder based on the unsupervised loss.