Patent ID: 11928598
Assignee: ALIBABA GROUP HOLDING LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G

Claim 6:
7. A neural network processing system, comprising:
a network connection; and
a plurality of heterogeneous computation units (HCUs) each comprising a computing unit and communicatively coupled to the network connection, the plurality of HCUs being configured to:
compute a plurality of first gradients from a plurality of first samples, wherein the plurality of first gradients are aggregated to generate an aggregated gradient;
compute a plurality of second gradients from a plurality of second samples;
aggregate, at each of the plurality of HCUs, the aggregated gradient of the plurality of first gradients with the each HCU's corresponding second gradient among the plurality of second gradients to generate a local gradient update for the each HCU; and
update, at each of the plurality of HCUs, a local copy of a neural network with the corresponding local gradient update for the each HCU.