Patent ID: 11941528
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 16:
17. An apparatus, comprising:
a neural network processor;
a hardware interface;
an integrated circuit comprising a weight gradients splitter and an exchange processor; and
a controller configured to:
control the neural network processor configured to perform backward propagation computations for a second layer of a neural network to generate second weight gradients;
control the weight gradients splitter to split the second weight gradients into portions;
control, via the exchange processor, the hardware interface to exchange a first portion of the second weight gradients with a second computer system separate from the apparatus, wherein the apparatus is a first computer system;
control the neural network processor to perform backward propagation computations for a first layer of the neural network to generate first weight gradients when the exchange of the first portion of the second weight gradients is underway, the first layer being a lower layer than the second layer in the neural network;
after transmission of the first portion of the second weight gradients completes and before transmitting remaining portions of the second weight gradients, control, via the exchange processor, the hardware interface to transmit all of the first weight gradients to the second computer system, such that the first weight gradients are prioritized over the second weight gradients; and
after the transmission of the first weight gradients is complete, control, via the exchange processor, the hardware interface to transmit the remaining portions of the second weight gradients to the second computer system.