Patent ID: 11875247
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method for processing data in a neural network acceleration engine having a plurality of neural network accelerators, the method comprising:
using direct memory access:
loading a set of weight values associated with a neural network model from an external memory of the neural network acceleration engine into a first state buffer of a first neural network accelerator of the plurality of neural network accelerators; and
for each neural network accelerator in the plurality of neural network accelerators other than the first neural network accelerator, sequentially loading the set of weight values that was stored in the external memory from a previous state buffer of a previous neural network accelerator into a state buffer of the neural network accelerator;

splitting, by a runtime driver, a set of input data into batches of data for the plurality of neural network accelerators;
for each batch of data in the batches of data, loading the batch of data into a corresponding neural network accelerator; and
for each neural network accelerator in the plurality of neural network accelerators, loading the set of weight values from the state buffer of the neural network accelerator into a processing engine array of the neural network accelerator, and performing neural network computations on a corresponding batch of data with the set of weights values loaded into the neural network accelerator,
wherein for each neural network accelerator in the plurality of neural network accelerators other than a last neural network accelerator, the set of weight values remains stored in the state buffer until:
the set of weight values has been loaded into the processing engine array of the corresponding neural network accelerator; and
the set of weight values has been loaded from the corresponding neural network accelerator into a next neural network accelerator.