Patent ID: 11941528
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 2:
3. The method of claim 1, wherein exchange of the portion of the second layer weight gradients with the second worker node comprises:
transmitting, to the second worker node, the first portion of the second layer weight gradients generated at the first worker node;
receiving, by the first worker node and from the second worker node, a first portion of the second layer weight gradients generated at the second worker node;
determining, by the first worker node, averaged second weight gradients based on averaging the first portion of the second layer weight gradients generated at the first worker node with the first portion of the second layer weight gradients generated at the second worker node; and
updating the weights for the second neural network layer using the averaged second weight gradients.