Patent ID: 11941528
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 9:
10. The non-transitory computer readable medium of claim 9, further storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to:
divide the first weight gradients into a plurality of portions;
assign each portion of the first weight gradients to each of a plurality of second exchange tasks;
store the plurality of second exchange tasks sequentially in the buffer after the plurality of first exchange tasks; and
responsive to determining that the transmission of the portion of the second weight gradients and the reception of the third weight gradients is complete:
retrieve each of the plurality of second exchange tasks from the buffer sequentially; and
execute the each of the plurality of second exchange tasks based on causing the hardware interface to transmit each portion of the first weight gradients assigned to the each of the plurality of second exchange tasks;

retrieve each of the remaining first exchange tasks from the buffer sequentially; and
execute the each of the remaining first exchange tasks based on causing the hardware interface to transmit each portion of the second weight gradients assigned to the each of the remaining first exchange tasks.