Patent ID: 11860969
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method performed by one or more computers, the method comprising:
receiving an input sequence of elements to be transformed into an output sequence of elements according to a learned transformation, each element of the input sequence having a respective initial input representation;
performing an encoding process including repeatedly revising the input representations in parallel using a same series of encoding operations for each of multiple time steps;
initializing a target sequence of elements each having a respective initial target representation;
performing a decoding process including repeatedly revising the target representations in parallel using two-stage self-attention, wherein a second stage of the two-stage self-attention uses an output generated by the encoding process after repeatedly revising the input representations; and
generating, from a final version of the revised target representations, the output sequence of elements representing the learned transformation of the input sequence of elements.