Patent ID: 11908458
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 15:
16. A computer-implemented method for customizing a recurrent neural network transducer (RNN-T), comprising:
synthesizing first domain audio data from first domain text data from a first domain;
feeding the synthesized first domain audio data into a trained encoder of the recurrent neural network transducer (RNN-T) having an initial condition, and updating the encoder by inputting both the synthesized first domain audio data and the first domain text data by encoding the synthesized first domain audio data into acoustic embedding, a1, wherein the acoustic embedding, a1, compresses the synthesized first domain audio data into a smaller feature space;
feeding the acoustic, embedding, a1, to a joiner;
synthesizing second domain audio data from second domain text data from a second domain;
feeding the synthesized second domain audio data into the updated encoder, wherein the updated encoder encodes the synthesized second domain audio data into acoustic embedding, b1, wherein the acoustic embedding, b1, compresses the synthesized second domain audio data into a smaller feature space;
feeding an output sequence from the joiner into a predictor of the recurrent neural network transducer (RNN-T), and updating the predictor by inputting both the synthesized second domain audio data and the second domain text data;
encoding a sequence of labels from the ins, and second domain text data into a text embedding using the predictor, and combining outputs of the encoder and the predictor with the synthesized first and second domain audio data using a joiner; and
resetting the updated encoder to a pre-customized state by restoring weights on the updated encoder to the initial condition including a state trained on the first domain.