Patent ID: 11922292
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 17:
18. A non-transitory machine-readable storage device for implementing a neural network having multiple neural network layers on a circuit used to perform neural network computations and for storing instructions that are executable by a processing device to cause performance of operations comprising:
providing, from a first memory, data used to generate an output for a neural network layer;
storing vectors of values at a first processor core of the circuit using a first vector memory of the first processor core, wherein the first vector memory is located within the first processor core and configured to store first vector values derived from the data provided by the first memory, wherein the first processor core further comprises a first vector register located within the first processor core and configured to at least load data from or store data to the first vector memory;
storing vectors of values at a second processor core of the circuit using a second vector memory of the second processor core, wherein the second vector memory is located within the second processor core and configured to store second vector values derived from the data provided by the first memory, wherein the second processor core further comprises a second vector register located within the second processor core and configured to at least load data from or store data to the second vector memory;
routing, using a first direct memory access (DMA) data path of a shared memory in the circuit, data communications comprising at least the first vector values between the shared memory and the first vector memory included in the first processor core, wherein the shared memory and the first memory are communicatively coupled by direct memory access (DMA);
routing, using a second direct memory access (DMA) data path of the shared memory in the circuit, data communications comprising at least the second vector values between the shared memory and the second vector memory included in the second processor core;
routing, using a first load-store data path of the shared memory, data communications comprising third vector values between the shared memory and the first vector register included in the first processor core;
routing, using a second load-store data path of the shared memory, data communications comprising fourth vector values between the shared memory and the second vector register included in the second processor core, wherein the shared memory further comprises a software-controlled staging resource that is formed from a subset of memory resources of the shared memory;
managing, using the software-controlled staging resource, data flows from the first memory to the first vector register of the first processor core and data flows from the first memory to the second vector register of the second processor core;
wherein the software-controlled staging resource is a first-in-first-out (FIFO) memory structure along a load section of the first load-store data path or the second load-store data path, and wherein the FIFO memory structure is configured to temporarily store a vector of values received from other memory locations of the shared memory for a threshold number of processor cycles and to subsequently route the vector of values from the FIFO memory structure to the first vector register of the first processor core or the second vector register of the second processor core; and
generating, by a matrix computation unit within the first processor core or the second processor core, accumulated values corresponding to the output for the neural network layer using the respective first and third vector values that are routed to the matrix computation unit in parallel along the first load-store data path and the first DMA data path of the shared memory, respectively, in response to performing a subset of the computations to generate the output for the neural network layer; wherein the software-controlled staging resource manages data flows between the first memory and the matrix computation unit, wherein the data flows comprise vector arrays that are derived from the data provided by the first memory.