Patent ID: 11875248
Assignee: IMAGINATION TECHNOLOGIES LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method for implementing a deep neural network in multicore hardware, the deep neural network including a plurality of layers, the multicore hardware comprising at least a first core and a second core, the method comprising:
defining a plurality of layer groups, including at least a first layer group and a second layer group, each layer group comprising one or more consecutive layers of the neural network, wherein output data of the first layer group comprises input data for the second layer group;
providing, in one or more first memory devices, input data for the first layer group, the input data comprising a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of the first layer group, and one or more non-traversed dimensions;
determining a size of the input data in a first dimension among the one or more traversed dimensions,
if the size in the first dimension is greater than a threshold, evaluating the first and second layer groups by depth-first tiling, comprising:
splitting the input data for the first layer group into at least a first tile and a second tile, along the first dimension, each tile comprising a plurality of data elements in each of the one or more traversed dimensions;
using the first core, evaluating the first layer group based on the first tile, to generate a first output tile of the first layer group;
using the second core, evaluating the first layer group based on the second tile, to generate a second output tile of the first layer group;
storing said first output tile and said second output tile of the first layer group in one or more second memory devices, as input data for the second layer group;
using the first core, evaluating the second layer group based on the first output tile of the first layer group, to generate a first output tile of the second layer group;
using the second core, evaluating the second layer group based on the second output tile of the first layer group, to generate a second output tile of the second layer group, and

if the size in the first dimension is not greater than the threshold, evaluating the first and second layer groups in multiple passes, one layer group at a time, comprising:
splitting the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension;
using the first core, evaluating the first layer group for the first pass, to generate a first portion of output data;
using the second core, evaluating the first layer group for the second pass, to generate a second portion of output data;
producing the output data of the first layer group, comprising combining the first portion of output data and the second portion of output data; and
storing the output data of the first layer group in the one or more second memory devices, as input data for the second layer group.