Patent ID: 11941528
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 6:
7. A non-transitory computer readable medium storing instructions that, when executed by one or more hardware processors of a first computer system, cause the one or more hardware processors to:
perform, at the first computer system, backward propagation computations for a second layer of a neural network to generate second weight gradients;
split, at the first computer system, the second weight gradients into portions;
cause a hardware interface to exchange a first portion of the second weight gradients with a second computer system separate from the first computer system;
perform, at the first computer system, backward propagation computations for a first layer of the neural network to generate first weight gradients when the exchange of the first portion of the second weight gradients is underway, the first layer being a lower layer than the second layer in the neural network;
after transmission of the first portion of the second weight gradients completes and before transmitting remaining portions of the second weight gradients, cause the hardware interface to transmit all of the first weight gradients to the second computer system, such that the first weight gradients are prioritized over the second weight gradients; and
after the transmission of the first weight gradients completes, cause the hardware interface to transmit the remaining portions of the second weight gradients to the second computer system.