Patent ID: 11954594
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method for training a sequence generation model, wherein:
for each particular position after an initial position in an output order of a target sequence, the sequence generation model is configured (i) to receive an input that is based on a set of preceding output scores that the sequence generation model generated for a prediction at a preceding position in the output order and (ii) to generate a current set of output scores for the particular position in the output order, and
the current set of output scores for the particular position in the output order comprises a respective score for each of a plurality of possible predictions for the particular position, and
the method comprising:
obtaining a plurality of training data pairs for the sequence generation model, each training data pair comprising a training input and a training target sequence for the training input, each training target sequence comprising a respective plurality of true outputs arranged according to an output order; and
training the sequence generation model on the training data pairs, comprising, for each training data pair and for each particular position after an initial position in the output order of the training target sequence of the training data pair, selecting an input to provide to the sequence generation model for generating an output at the particular position in the output order of the training target sequence, wherein the input is selected from a group comprising (i) a non-predicted input that is based on the true output from a preceding position in the output order of the training target sequence of the training data pair, and (ii) a predicted input that is based on a set of preceding output scores that the sequence generation model generated at a preceding model.