Patent ID: 11860969
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 11:
12. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving an input sequence of elements to be transformed into an output sequence of elements according to a learned transformation, each element of the input sequence having a respective initial input representation;
performing an encoding process including repeatedly revising the input representations in parallel using a same series of encoding operations for each of multiple time steps;
initializing a target sequence of elements each having a respective initial target representation;
performing a decoding process including repeatedly revising the target representations in parallel using two-stage self-attention, wherein a second stage of the two-stage self-attention uses an output generated by the encoding process after repeatedly revising the input representations; and
generating, from a final version of the revised target representations, the output sequence of elements representing the learned transformation of the input sequence of elements.