Patent ID: 11928598
Assignee: ALIBABA GROUP HOLDING LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G

Claim 14:
15. A non-transitory computer readable storage media storing a set of instructions that are executable by one or more processing devices in a neural network processing system comprising a plurality of heterogeneous computation units (HCUs) to cause the neural network processing system to perform a method comprising:
computing a plurality of first gradients from a plurality of first samples;
aggregating the plurality of first gradients to generate an aggregated gradient;
computing a plurality of second gradients from a plurality of second samples;
aggregating, at each of the plurality of HCUs, the aggregated gradient of the plurality of first gradients with the each HCU's corresponding second gradient among the plurality of second gradients to generate a local gradient update for the each HCU; and
updating, at each of the plurality of HCUs, a local copy of a neural network with the corresponding local gradient update for the each HCU.