Patent Document ID: 10140572
Application ID: 14750277
Patent Status: 1

Claim One:
1. A method of performing neural network processes, the method comprising: receiving, at a set of N input buffers, a batch of input data for accelerated processing of a neural network evaluation, wherein each input buffer provides a queue with a queue depth of at least two, where N is a number of available parallel streams of a field programmable gate array (FPGA), wherein the batch of input data comprises at least two observation vectors, one of the at least two observation vectors being loaded at a first position in the queue of the set of N input buffers and a second of the at least two observation vectors being loaded at a second position in the queue of the set of N input buffers; loading the FPGA with a first layer set of weights for the neural network evaluation from an external memory; and applying, within the FPGA, the first layer set of weights to the batch of the input data to generate intermediates, wherein while the one of the observation vectors is being processed, the second of the observation vectors moves from the second position to the first position in the queue of the input buffers and the intermediates generated from the one of the observation vectors at the first position in the queue are re-queued for next layer processing; and wherein after the second of the at least two feature vectors is processed, the intermediates generated from the second of the observation vectors are re-queued for the next layer processing.