Patent ID: 11922951
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 8:
9. A client device comprising:
one or more processors, and
memory configured to store instructions that, when executed by the one or more processors, cause the one or more processors to perform a method that includes:
generating a speaker embedding for a human speaker, wherein generating the speaker embedding for the human speaker comprises:
processing one or more instances of speaker audio data corresponding to the human speaker using a trained speaker embedding model, and
generating the speaker embedding based on one or more instances of output each generated based on processing a respective of the one or more instances of speaker audio data using the trained speaker embedding model;

receiving audio data that captures one or more utterances of the human speaker and that also captures one or more additional sounds that are not from the human speaker;
generating a refined version of the audio data, wherein the refined version of the audio data isolates the one or more utterances of the human speaker from the one or more additional sounds that are not from the human speaker, and wherein generating the refined version of the audio data comprises:
processing the audio data using a frequency transformation to generate an audio spectrogram, wherein the audio spectrogram is a frequency domain representation of the audio data;
processing the audio spectrogram and the speaker embedding using a trained voice filter model to generate a predicted mask,
wherein the predicted mask isolates the one or more utterances of the human speaker from the one or more additional sounds in the audio spectrogram,
wherein the trained voice filter model comprises a convolutional neural network portion, a recurrent neural network portion, and a fully connected feed-forward neural network portion, and
wherein processing the audio spectrogram to generate the predicted mask comprises:
 processing the audio spectrogram using the convolutional neural network portion of the trained voice filter model to generate convolutional output;
 processing the speaker embedding and the convolutional output using the recurrent neural network portion of the trained voice filter model to generate recurrent output; and
 processing the recurrent output using the fully connected feed-forward neural network portion of the trained voice filter model to generate the predicted mask;

generating a masked spectrogram by processing the audio spectrogram using the predicted mask, wherein the masked spectrogram captures the one or more utterances of the human speaker and not the one or more additional sounds; and
generating the refined version of the audio data by processing the masked spectrogram using an inverse of the frequency transformation.