Patent ID: 11966766
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A data processing system, comprising:
one or more host processing devices, the one or more host processing devices supporting instantiation of a plurality of virtual machines;
a first set of virtual machines comprising one or more of the plurality of virtual machines, the first set of virtual machines running one or more worker processes, each worker process operating on a respective data set to produce a respective gradient; and
a second set of virtual machines comprising one or more of the plurality of virtual machines, the second set of virtual machines running one or more reducer processes that operate on at least a portion of each respective gradient produced by each worker process to produce an aggregated gradient, the second set of virtual machines being configured such that computations or communications associated with the one or more reducer processes are balanced across the one or more reducer processes;
wherein the second set of virtual machines are stateless and the one or more reducer processes cause the aggregated gradient to be broadcasted to each worker process
wherein a first total bandwidth associated with use of the first set of virtual machines is equal to a second total bandwidth associated with use of the second set of virtual machines.