Patent ID: 12242416

DETAILED DESCRIPTION

FIG.1is a schematic diagram that illustrates an exemplary heterogeneous architecture100for combined deep learning and general purpose computing. The architecture100includes a central processing unit (CPU) core105and an accelerator110with data communication managed by a direct memory access (DMA) engine115. However, there remain the challenges of low utilization of processing element (PE) cores and large latency due to the CPU workload and data movement across processing cores. For example, in an end-to-end deep learning task, the accelerator110may often be utilized at only approximately 30-50% with the rest of time waiting for CPU processing and data movement between the CPU105and accelerator110cores.

FIG.2is a schematic diagram that illustrates an architecture of an exemplary systolic neural CPU (SNCPU)200. The SNCPU200may fuse operations of a conventional CPU (e.g., CPU105) and a systolic convolutional neural network (CNN) accelerator (e.g., accelerator110) in a single SNCPU core205. The SNCPU core205may include a RISC-V PE array210, an SRAM cache215, a level 2 memory (L2)220, a rectified linear unit (ReLU) pooling scaling control225, a register file (RF) unit230, and an accumulator235. A memory control unit240may control data transfers via an interface or data bus245between the SNCPU core205and one or more global memory units250.

Benefits of the SNCPU200may include, but are not limited to, the following:1) the architecture of the SNCPU200may be flexibly reconfigured into a multi-core RISC-V CPU (e.g., a CPU based on an instruction set architecture (ISA) rooted in reduced instruction set computer (RISC) principles) or a systolic CNN accelerator, leading to PE utilization of over 95% for end-to-end operation;2) with an overhead of less than 10%, the CNN accelerator may be reconfigured into a 10-core RISC-V CPU to improve throughput significantly compared with a conventional heterogeneous architecture having a CPU and an accelerator;3) with a special bi-directional dataflow, expensive data movement for inter-layer pre/post-processing across cores may be avoided; and4) experimental demonstrations of the SNCPU200through a 65 nm test chip show 39% to 64% latency improvement and 0.65 to 1.8 tera-operations-per-second-per-Watt (TOPS/W) energy efficiency on end-to-end image-classification tasks.

FIG.3Ais a schematic diagram that illustrates a top-level architecture of an exemplary systolic neural CPU (SNCPU)300. The SNCPU300may be an example of the SNCPU200. The SNCPU300includes a reconfigurable systolic array305of PEs organized into rows and columns. As illustrated, the SNCPU300includes ten (10) rows (Row0. . . Row9) and ten (10) columns of PEs (PE0. . . PE9), but this should not be construed as limiting, as various examples of the SNCPU300may include more or fewer rows and/or columns of PEs in the reconfigurable systolic array.FIG.3Bis a schematic diagram that illustrates the exemplary SNCPU300in a RISC-V CPU configuration mode.FIG.3Cis a schematic diagram that illustrates the exemplary SNCPU300in a systolic array CNN configuration mode.FIG.3Dis a schematic diagram that illustrates the exemplary SNCPU300in a hybrid RISC-V CPU systolic array configuration mode. The reconfigurable systolic array305, a 10×10 array of PEs as shown inFIG.3A, may serve as the central computing tiles. Each lane of the PE array (e.g., each row or each column of PEs) may be configured as either systolic multiplication-accumulate (MAC) operations for the CNN accelerator or CPU pipeline stages.

In RISC-V CPU configuration mode (i.e., CPU mode) shown inFIG.3B, each row or column of ten (10) PEs may be used to realize RISC-V pipelines. Associated static random access memory (SRAM) banks may also be reconfigured for both purposes. Although data may stay mostly local within the reconfigurable SRAM banks, level two (L2) SRAM banks (row L2 memory and column L2 memory) may also be included in CPU mode to enable data exchange between different CPU cores during data processing. Instruction caches (e.g., Instr. Mem. modules at the left side of each row and at the top of each column) may also be included in CPU mode.

The systolic array CNN configuration mode (i.e., accelerator mode) shown inFIG.3Cmay support typical systolic dataflow with weight-stationary operations. In accelerator mode, an accumulator (ACT module) for each row or column may provide additional single instruction, multiple data (SIMD) support for pooling, rectified linear unit (ReLU) functionality, and accumulation. The ACT modules may also support activation and scaling.

In an available hybrid RISC-V CPU systolic array configuration mode (i.e., hybrid RISC-V and accelerator mode) shown inFIG.3D, half of the PE cores in the reconfigurable systolic array305may be configured into CPU mode and the other half of the PE cores may be configured into the systolic CNN accelerator mode. The hybrid mode features bi-directional dataflow. Each row and/or column may be configured as one RISC-V pipeline core. The AOMEM modules at the bottom of each column and the right of each row in hybrid mode may be reconfigurable as activation memory (A_mem), output memory (O_mem), or data cache modules.

FIG.4Ais a schematic diagram that illustrates one row400of the exemplary reconfigurable systolic array305.FIG.4Bis a schematic diagram that illustrates one row400of the exemplary reconfigurable systolic array305reconfigured as one exemplary RISC-V CPU core. Either a row or a column of PEs may be reconfigured as one RISC-V CPU core, in various examples. The RISC-V CPU core may include a 32b RISC-V CPU pipeline constructed from the systolic array305of PEs. Similar to a typical accelerator design, each PE in the systolic array305may include a simple pipelined multiplication-accumulate (MAC) unit with 8b-wide inputs and weights and 32b at accumulation output. As shown inFIG.4B, when the row400of PEs is configured as a RISC-V CPU core, the A_mem or AOMEM (seeFIG.3A) memory unit at the far right of the row400may be reconfigured as a data cache Dcache. As the Dcache is bidirectionally communicatively coupled with the memory controller Mem Ctrl, WB PE9, and MEM PE8of the RISC-V CPU core of row400, any data stored in the memory unit Dcache (including data inherited from the A_mem or AOMEM memory elements prior to being configured as a data cache Dcache) may be transferred from the Dcache at the end of the RISC-V CPU pipeline to the register file RF to be operated upon in the EX stage according to the instruction(s) fetched in the IF stage and decoded in the ID stage.

FIG.4Cis a schematic diagram that illustrates one PE of the exemplary RISC-V CPU core shown inFIG.4Bconfigured for program counter (PC) calculations. As shown inFIG.4C, the very first PE in a row (e.g., PE0) or column reconfigured as one exemplary RISC-V CPU core may reuse the MAC's adder and 32b registers as PC for the instruction cache address.

FIG.4Dis a schematic diagram that illustrates two PEs of the exemplary RISC-V CPU core shown inFIG.4Bconfigured for instruction fetch (IF) operations. Two PEs (PE1and PE2) are used as the IF stage for instruction fetch with a reuse of the internal 32b register and 8b input registers.

FIG.4Eis a schematic diagram that illustrates a first (PE3) of two PEs of the exemplary RISC-V CPU core shown inFIG.4Bconfigured for instruction decode (ID) stage operations.FIG.4Fis a schematic diagram that illustrates a second (PE4) of two PEs of the exemplary RISC-V CPU core shown inFIG.4Bconfigured for ID stage operations. Two PEs are reconfigured into the instruction decode/decoder (ID) stage where the logic in the 8b multiplier and 32b adder are reused to generate control signals by performing numerical/logical operations with the op-code or func-code of instructions.

FIG.4Gis a schematic diagram that illustrates an EX stage ALU reconfigured from some portions of three PEs (PE5, PE6, PE7) of the exemplary RISC-V CPU core shown inFIG.4B, andFIG.4His a schematic diagram that illustrates an EX stage branch reconfigured from other portions of the three PEs (PE5, PE6, PE7) of the exemplary RISC-V CPU core shown inFIG.4B. Three PEs (PE5, PE6, PE7) may be combined into the execution (EX) stage, including one PE serving as arithmetic logic unit (ALU) with additional logic for Boolean operations and a shifter, one PE to generate new instruction cache address for branches, and one PE used as registers to pass the execution results.

FIG.4Iis a schematic diagram that illustrates one PE (PE8) of the exemplary RISC-V CPU core shown inFIG.4Bconfigured for memory (MEM) stage operations.FIG.4Jis a schematic diagram that illustrates one PE (PE9) of the exemplary RISC-V CPU core shown inFIG.4Bconfigured for write-back (WB) stage operations. The last two PEs (PE8, PE9) of the exemplary RISC-V CPU core shown inFIG.4Bmay be reconfigured into the memory (MEM) stage and the write-back (WB) stage by reusing registers with additional multiplexer (MUX) logic. Registers may be reused for fetched data. Forwarding paths may also be added to support CPU data dependency.

FIG.5Ais a graph that illustrates exemplary PE logic utilization for the CPU pipeline of the RISC-V CPU core shown inFIG.4B.

FIG.5Bis a graph that illustrates exemplary total area overhead for the CPU pipeline of the RISC-V CPU core shown inFIG.4B.

FIG.5Cis a graph that illustrates exemplary accelerator power overhead for the CPU pipeline of the RISC-V CPU core shown inFIG.4B. With an emphasis on logic sharing, an exemplary PE logic reconfiguration for CPU mode may reuse 64% to 80% of the original PE logic for CPU construction, as shown inFIG.5A. In an example compared with the baseline original systolic CNN accelerator design, the area overhead to include CPU functions is 3.4% in the PE array, 6.4% in the memory (e.g., instruction and register file (RF)), and overall 9.8% for the whole processor, as shown inFIG.5B. Extensive clock gating may be used to eliminate redundant power consumption from the additional logic in both CPU and CNN modes. In an example, the power overhead for the CNN accelerator is about 15% compared with the baseline original systolic CNN accelerator design, as shown inFIG.5C.

The SNCPU architecture described herein may facilitate a majority of data to be retained inside the processor core, eliminating performing expensive data movement and using the DMA module. To enhance data locality, a special dataflow sequence for CNN operation may be adopted combining the two (2) configurable modes (CPU, accelerator) and two (2) directions (row-based and column-based). Four different resulting configurations for dataflow with activated modules are highlighted inFIGS.6-9.FIG.6is a schematic diagram that illustrates an exemplary reconfigurable systolic array of PEs of an SNCPU300reconfigured for a column-accelerator mode dataflow600.FIG.7is a schematic diagram that illustrates an exemplary reconfigurable systolic array of PEs of an SNCPU300reconfigured for a column-CPU mode dataflow700.FIG.8is a schematic diagram that illustrates an exemplary reconfigurable systolic array of PEs of an SNCPU300reconfigured for a row-accelerator mode dataflow800.FIG.9is a schematic diagram that illustrates an exemplary reconfigurable systolic array of PEs of an SNCPU300reconfigured for a row-CPU mode dataflow900.

The column-accelerator mode dataflow600shown inFIG.6may be the same as in a conventional weight-stationary systolic array. Each “AOMEM” SRAM bank in every row may be used as input memory610and each AOMEM bank in every column may serve as output memory620to store accumulated results. Activation630may be performed from right to left, input data may go through every PE in each row from right to left, and the accumulation640may be performed downward with results passing down in each column from ROW0to ROW9. Instruction caches may be gated during accelerator mode.

The column-CPU mode dataflow700shown inFIG.7may pass instructions from top instruction caches710downward through the pipeline715while the bottom AOMEM banks are reconfigured to data caches720, which facilitates one column of PEs to be reconfigured into one RISC-V CPU pipeline core.

In the row-accelerator mode800shown inFIG.8, the PEs may receive the inputs from bottom AOMEM banks (configured as input memory810) and store the results in the right AOMEM banks (configured as output memory820). The dataflow direction in the row-accelerator mode800may be an orthogonal direction dataflow as compared to the column-accelerator mode shown inFIG.6, with activation830going upward in each column from the last row to the first row and accumulation840going rightward in each row from the left to the right.

In the row-CPU mode dataflow900shown inFIG.9, every row may be configured as a 5-stage pipelined core, with every row's AOMEM banks serving as data cache910. Instructions may be passed through the pipeline920from left to right.

FIGS.10A-10Fillustrate a four-phase (4-phase) dataflow utilizing the four dataflow configurations of an SNCPU300described with reference toFIGS.6-9for end-to-end image classification tasks. In a conventional architecture, the DMA engine may be used to transfer input data from a CPU cache to a scratch pad of the accelerator; however, this use of the DMA engine to transfer data may be avoided in the 4-phase dataflow of the SNCPU300described herein.

In a first phase Step1, the SNCPU300may operate in row-CPU mode dataflow900to perform CPU pre-processing/inter-layer data processing including input-data preprocessing (e.g., image reshape, rotation, normalization, grayscale) for the CNN. In a second phase Step2, the SNCPU300may operate in column-accelerator mode dataflow600to perform DNN 1st layer (L1) processing with the data caches910from the first-phase CPU mode reused as input memory610for the second-phase CNN accelerator mode. In a third phase Step3, after the CNN accelerator finishes the entire layer of the CNN model, the SNCPU300may be reconfigured to column-CPU mode dataflow700to perform inter-layer data processing including data alignment, padding, duplication, and post-processing by directly using the data from the output memory620from the second-phase accelerator mode as already present in the data caches720of the third-phase CPU mode. In a fourth phase Step4, the SNCPU300may switch to row-accelerator mode dataflow800to perform DNN 2nd layer (L2) processing to process the second layer of the CNN by directly using the data from the data caches720of the third-phase CPU mode as already present in the input memory810of the fourth-phase accelerator mode. The 4-phase sequence of operations may repeat through the cycle from Step1, to Step2, to Step3, to Step4, and back to Step1again, etc., until all CNN layers are finished eliminating intermediate data transfer across cores. In addition, as the SNCPU300may be configured into ten (10) CPU cores, which together may perform ten (10) separate instructions at the same time, a significant improvement of CPU pre/post-processing may be achieved compared with a conventional CPU+CNN architecture. Using the four-phase cycle described with reference toFIGS.6,7,8,9and10A-10E, keeping data local within the SNCPU300while performing bidirectional data flow through the SNCPU300combined with reconfiguration of the SNCPU300and its PEs to perform different functions in each of four phases in a four-phase cycle eliminates some data transfers that a conventional CPU+CNN architecture performs.

FIGS.11A-11Dare graphs that illustrate end-to-end performance comparisons between exemplary experimental demonstrations of the SNCPU300and conventional DNN+CPU architectures. The experimental demonstrations of the SNCPU300include implementations of an end-to-end image classification operation using 8b quantized VGG16, ResNet18, 3-layer ELU models on CIFAR10, ImageNet and MNIST datasets. Results show 39%-to-64% improvement in latency compared with a conventional heterogeneous accelerator (DNN+CPU) architecture (e.g., Gemmini). As shown inFIG.11A, elimination of some idle cycles and elimination of data transfer (e.g., DMA data transfer) of the conventional DNN+CPU architecture by the SNCPU300play roles in reducing latency in the SNCPU300, although the SNCPU300does perform pre-processing in the row-CPU phase and performs data alignment and padding in the column-CPU phase. Results show that the 64% latency improvement breaks down as: 33% from 10-core CPU parallel processing and 31% from eliminated data movements. For workloads requiring less CPU or data movements, fewer benefits were observed as in the case for the MNIST dataset.

FIG.12is a diagram illustrating a floorplan of a 65 nm CMOS process technology exemplary SNCPU test chip1200. The SNCPU test chip1200may be an example of the SNCPU200and/or SNCPU300. The 65 nm SNCPU test chip1200was fabricated and tested at a nominal supply of 1.0V. The SNCPU test chip1200has dimensions of 2.07 mm by 2.16 mm for an accelerator area of 4.47 mm2. The floorplan of the SNCPU test chip1200includes a central PE array block with SRAM blocks on each of four sides of the central PE array block. Additionally, in the lower right corner of the SNCPU test chip1200floorplan, a control block, scan block, and DCO block are provided. The PE number is 100, arranged as a 10×10 array of PEs. The 10-core CPU consumes 589 mW, whereas the DNN consumes 116 mW of power. Eight bit integer precision (INT8) is used in this design. The nominal frequency at which the SNCPU test chip1200is clocked is 400 MHz. The supply voltage Vddis between 0.5 V and 1 V. An amount of static random access memory (SRAM) provided on the SNCPU test chip1200is 150 KB. Energy efficiency is 0.66 to 1.82 TOPS/W at INT8 precision over the supply voltage range from 1 V to 0.5 V.

FIGS.13A-Dare graphs that illustrate measurement results of the exemplary SNCPU test chip1200.FIG.13Ashows a power trace that illustrates the 4-phase operation with continuous core utilization above 95% in both CPU and accelerator mode. As shown inFIG.13A, power trace is near 0 mW and utilization is near 85% for the first ˜80 μs. From ˜80 μs to ˜200 μs, the SNCPU test chip1200is configured as 10 row-CPU cores for preprocessing, such as image reshaping, grayscale, rotation, normalization, and other image processing functions. During this preprocessing, the power trace shows about 600 mW of power consumed while utilization is at about 96%. From ˜200 μs to ˜260 μs, the SNCPU test chip1200is configured in column-DNN mode for performing a first layer convolution. During this column-DNN mode, the power trace shows between about 100-150 mW of power consumed while utilization is at about 99%. From ˜260 μs to ˜390 μs, the SNCPU test chip1200is configured as 10 column-CPU cores for data alignment, padding, movement, and duplication for the next layer. During this column-CPU phase, the power trace shows about 600 mW of power consumed while utilization is at about 96%. From ˜390 μs to ˜440 μs, the SNCPU test chip1200is configured in row-DNN mode for performing a second layer convolution. During this row-DNN mode, the power trace shows between ˜100-150 mW of power consumed while utilization is at about 99%.

FIG.13Bshows that the 10-core CPU mode consumes up to about 1 W of power as the supply voltage rises from 0.5 V to 1.2 V. In contrast, the DNN mode consumes up to about 300 mW of power as the supply voltage rises over the same range.

FIG.13Cshows that both CPU mode and DNN mode have a frequency ranging from a little over 0 MHz to 400 MHz as the supply voltage ranges from 0.5 V to 1.0 V, and then up to between about 500 MHz and 550 MHz when the supply voltage rises to 1.2 V.

FIG.13Dshows that the DNN energy efficiency drops from about 1.8 TOPS/W to 655 GOPS/W as the supply voltage increases from 0.5 V to 1.0 V, and then decreases further to about 400 GOPS/W as the supply voltage increases further to 1.2 V. These measurements were performed using 8 bit (8b) integer precision. In summary, 0.66-to-1.8 TOPS/W energy efficiency at 8b integer precision for CNN is achieved over the supply voltage range from 1.0 V to 0.5 V.

FIG.14is a table that shows a comparison of the exemplary SNCPU test chip1200shown inFIG.12with prior works. The table shows that the comparisons are made between the different works all fabricated in 65 nm process technology. The comparisons ofFIG.14show that the SNCPU test chip1200has the lowest power consumption at 116 mW (CNN) compared to 241 mW, 278 mW, and 279 mW, respectively, for the other compared works of MICRO2020, Eyeriss, and DNPU.

Compared with a prior reconfigurable binary neural network (BNN)-based design by the inventors in collaboration with others, the SNCPU test chip1200described herein converts a commonly used 8b systolic CNN accelerator into 10 CPU cores offering significantly higher performance and a broader set of use cases. In comparison with a conventional CNN+CPU architecture, a latency improvement of 39% to 64% is observed in the SNCPU test chip1200described herein.

FIG.15is a flowchart that illustrates an exemplary process1500of performing deep neural network processing and computing processing in a two-dimensional systolic array of reconfigurable processing elements. In some examples, one or more process blocks ofFIG.15may be performed by electronic circuitry and/or a computing device.

As shown inFIG.15, process1500may include configuring the two-dimensional systolic array of reconfigurable processing elements (PE's) into a row-CPU mode where the two-dimensional systolic array is configured to perform standard computations on input data (block1502). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a row-CPU mode where the two-dimensional systolic array is configured to perform standard computations on input data, as described above.

As also shown inFIG.15, process1500may include receiving input data by the two-dimensional systolic array configured into row-CPU mode (block1504). For example, electronic circuitry and/or a computing device may receive input data by the two-dimensional systolic array configured into row-CPU mode, as described above.

As further shown inFIG.15, process1500may include performing standard computations on the input data by the two-dimensional systolic array (block1506). For example, electronic circuitry and/or a computing device may perform standard computations on the input data by the two-dimensional systolic array, as described above.

As also shown inFIG.15, process1500may include saving results of the row-CPU computations in memory elements local to the two-dimensional systolic array (block1508). For example, electronic circuitry and/or a computing device may save results of the row-CPU computations in memory elements local to the two-dimensional systolic array, as described above.

As further shown inFIG.15, process1500may include configuring the two-dimensional systolic array of reconfigurable processing elements into a column-accelerator mode where the two-dimensional systolic array is configured to perform deep neural network processing on input data (block1510). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a column-accelerator mode where the two-dimensional systolic array is configured to perform deep neural network processing on input data, as described above.

As also shown inFIG.15, process1500may include performing column-accelerator mode neural network computations starting from the results from the row-CPU mode computations in the local memory elements where the results were saved as input data by the two-dimensional systolic array configured into column-accelerator mode (block1512). For example, electronic circuitry and/or a computing device may perform column-accelerator mode neural network computations starting from the results from the row-CPU mode computations in the local memory elements where the results were saved as input data by the two-dimensional systolic array configured into column-accelerator mode, as described above.

As further shown inFIG.15, process1500may include saving results of the column-accelerator mode neural network computations in memory elements local to the two-dimensional systolic array (block1514). For example, electronic circuitry and/or a computing device may save results of the column-accelerator mode neural network computations in memory elements local to the two-dimensional systolic array, as described above.

As also shown inFIG.15, process1500may include configuring the two-dimensional systolic array of reconfigurable processing elements into a column-CPU mode where the two-dimensional systolic array is configured to perform standard computations on input data (block1516). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a column-cpu mode where the two-dimensional systolic array is configured to perform standard computations on input data, as described above.

As further shown inFIG.15, process1500may include receiving input data by the two-dimensional systolic array configured into column-CPU mode (block1518). For example, electronic circuitry and/or a computing device may receive input data by the two-dimensional systolic array configured into column-CPU mode, as described above.

As also shown inFIG.15, process1500may include performing standard computations on the input data by the two-dimensional systolic array (block1520). For example, electronic circuitry and/or a computing device may perform standard computations on the input data by the two-dimensional systolic array, as described above.

As further shown inFIG.15, process1500may include saving results of the column-CPU computations in memory elements local to the two-dimensional systolic array (block1522). For example, electronic circuitry and/or a computing device may save results of the column-CPU computations in memory elements local to the two-dimensional systolic array, as described above.

As also shown inFIG.15, process1500may include configuring the two-dimensional systolic array of reconfigurable processing elements into a row-accelerator mode where the two-dimensional systolic array is configured to perform deep neural network processing on input data (block1524). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a row-accelerator mode where the two-dimensional systolic array is configured to perform deep neural network processing on input data, as described above.

As further shown inFIG.15, process1500may include performing row-accelerator mode neural network computations starting from the results from the column-CPU mode computations in the local memory elements where the results were saved as input data by the two-dimensional systolic array configured into row-accelerator mode (block1526). For example, electronic circuitry and/or a computing device may perform row-accelerator mode neural network computations starting from the results from the column-CPU mode computations in the local memory elements where the results were saved as input data by the two-dimensional systolic array configured into row-accelerator mode, as described above.

As also shown inFIG.15, process1500may include saving results of the row-accelerator mode neural network computations in memory elements local to the two-dimensional systolic array (block1528). For example, electronic circuitry and/or a computing device may save results of the row-accelerator mode neural network computations in memory elements local to the two-dimensional systolic array, as described above.

Process1500may include additional exemplary operations, such as any single exemplary operation or any combination of exemplary operations described below and/or in connection with one or more other processes described elsewhere herein. In a first additional example, process1500further includes determining if any neural network layers of the two-dimensional systolic array have any intermediate data transfer across cores remaining to be eliminated, and while intermediate data transfer across cores remains to be eliminated, continuing cycling through operations of the process1500in a circular sequence from row-CPU mode operations, to column-accelerator mode operations, to column-CPU mode operations, and to row-accelerator mode operations (block1530).

In a second additional example, alone or in combination with the first additional example, process1500may include causing data to flow in a direction during the row-accelerator mode neural network computations that is orthogonal to a direction data flows during the column-accelerator mode neural network computations.

In a third additional example, alone or in combination with the first and second additional examples of process1500, performing neural network computations may include performing systolic dataflow with weight-stationary operations, and performing standard computations by the two-dimensional systolic array may include performing computations by a sequence of RISC-V pipeline stages configured from the processing elements of the two-dimensional systolic array.

AlthoughFIG.15shows example blocks of process1500, in some examples, process1500may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted inFIG.15. Additionally, or alternatively, two or more of the blocks of process1500may be performed in parallel.

FIG.16is a flowchart that illustrates an exemplary process1600of performing deep neural network processing and computing processing in a two-dimensional systolic array of reconfigurable processing elements. In some examples, one or more process blocks ofFIG.16may be performed by electronic circuitry and/or a computing device.

As shown inFIG.16, process1600may include configuring the two-dimensional systolic array of reconfigurable processing elements into a row-CPU mode where each row of the two-dimensional systolic array includes a RISC-V pipeline core (block1602). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a row-CPU mode where each row of the two-dimensional systolic array includes a RISC-V pipeline core, as described above.

As also shown inFIG.16, process1600may include performing computing processing of each RISC-V pipeline core row from a leftmost column toward a rightmost column of the two-dimensional systolic array (block1604). For example, electronic circuitry and/or a computing device may perform computing processing of each RISC-V pipeline core row from a leftmost column toward a rightmost column of the two-dimensional systolic array, as described above.

As further shown inFIG.16, process1600may include storing results of the RISC-V pipeline core row processing in a right set of memory elements to the right of the rightmost column of the two-dimensional systolic array (block1606). For example, electronic circuitry and/or a computing device may store results of the RISC-V pipeline core row processing in a right set of memory elements to the right of the rightmost column of the two-dimensional systolic array, as described above.

As also shown inFIG.16, process1600may include configuring the two-dimensional systolic array of reconfigurable processing elements into a column-accelerator mode where the reconfigurable processing elements are configured as DNN processing elements (block1608). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a column-accelerator mode where the reconfigurable processing elements are configured as DNN processing elements, as described above.

As further shown inFIG.16, process1600may include performing column-accelerator mode activation by processing neural network data flowing from the right set of memory elements on the rightmost side of the rows of the two-dimensional systolic array leftward toward a leftmost column of the two-dimensional systolic array (block1610). For example, electronic circuitry and/or a computing device may perform column-accelerator mode activation by processing neural network data flowing from the right set of memory elements on the rightmost side of the rows of the two-dimensional systolic array leftward toward a leftmost column of the two-dimensional systolic array, as described above.

As also shown inFIG.16, process1600may include performing column-accelerator mode accumulation by processing neural network data flowing from the topmost row of the two-dimensional systolic array toward a bottom set of memory elements below the bottommost row of the two-dimensional systolic array (block1612). For example, electronic circuitry and/or a computing device may perform column-accelerator mode accumulation by processing neural network data flowing from the topmost row of the two-dimensional systolic array toward a bottom set of memory elements below the bottommost row of the two-dimensional systolic array, as described above.

As further shown inFIG.16, process1600may include storing results of the column-accelerator mode accumulation in the bottom set of memory elements below the bottommost row of the two-dimensional systolic array (block1614). For example, electronic circuitry and/or a computing device may store results of the column-accelerator mode accumulation in the bottom set of memory elements below the bottommost row of the two-dimensional systolic array, as described above.

As also shown inFIG.16, process1600may include configuring the two-dimensional systolic array of reconfigurable processing elements into a column-CPU mode where each column of the two-dimensional systolic array includes a RISC-V pipeline core (block1616). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a column-CPU mode where each column of the two-dimensional systolic array includes a RISC-V pipeline core, as described above.

As further shown inFIG.16, process1600may include performing computing processing of each RISC-V pipeline core column from a topmost row toward a bottommost row of the two-dimensional systolic array (block1618). For example, electronic circuitry and/or a computing device may perform computing processing of each RISC-V pipeline core column from a topmost row toward a bottommost row of the two-dimensional systolic array, as described above.

As also shown inFIG.16, process1600may include storing results of the RISC-V pipeline core column processing in the bottom set of memory elements below the bottommost row of the two-dimensional systolic array (block1620). For example, electronic circuitry and/or a computing device may store results of the RISC-V pipeline core column processing in the bottom set of memory elements below the bottommost row of the two-dimensional systolic array, as described above.

As further shown inFIG.16, process1600may include configuring the two-dimensional systolic array of reconfigurable processing elements into a row-accelerator mode where the reconfigurable processing elements are configured as DNN processing elements (block1622). For example, electronic circuitry and/or a computing device may configure the two-dimensional systolic array of reconfigurable processing elements into a row-accelerator mode where the reconfigurable processing elements are configured as DNN processing elements, as described above.

As also shown inFIG.16, process1600may include performing row-accelerator mode activation by processing neural network data flowing from the bottom set of memory elements below the bottommost row of the two-dimensional systolic array upward toward a topmost row of the two-dimensional systolic array (block1624). For example, electronic circuitry and/or a computing device may perform row-accelerator mode activation by processing neural network data flowing from the bottom set of memory elements below the bottommost row of the two-dimensional systolic array upward toward a topmost row of the two-dimensional systolic array, as described above.

As further shown inFIG.16, process1600may include performing row-accelerator mode accumulation by processing neural network data flowing from the leftmost column of the two-dimensional systolic array toward the right set of memory elements to the right of the rightmost column of the two-dimensional systolic array (block1626). For example, electronic circuitry and/or a computing device may perform row-accelerator mode accumulation by processing neural network data flowing from the leftmost column of the two-dimensional systolic array toward the right set of memory elements to the right of the rightmost column of the two-dimensional systolic array, as described above.

As also shown inFIG.16, process1600may include storing results of the row-accelerator mode accumulation in the right set of memory elements to the right of the rightmost column of the two-dimensional systolic array (block1628). For example, electronic circuitry and/or a computing device may store results of the row-accelerator mode accumulation in the right set of memory elements to the right of the rightmost column of the two-dimensional systolic array, as described above.

Process1600may include additional exemplary operations, such as any single operation or any combination of operations described below and/or in connection with one or more other processes described elsewhere herein. In a first additional example, process1600may include sending data stored in the right set of memory elements at a conclusion of a most recently completed row-accelerator mode accumulation process to corresponding register files of the respective RISC-V pipeline core rows and processing the sent data provided to the execution stage of the RISC-V pipeline core rows by the corresponding register files.

In a second additional example, alone or in combination with the first additional example, process1600may include sending data stored in the bottom set of memory elements at a conclusion of a most recently completed column-accelerator mode accumulation process to corresponding register files of the respective RISC-V pipeline core columns and processing the sent data provided to the execution stage of the RISC-V pipeline core columns by the corresponding register files.

In a third additional example, alone or in combination with the first and/or second additional examples, process1600may include using data stored in the bottom set of memory elements at a conclusion of a most recently completed column-CPU mode computing process as input data to the respective row-accelerator mode columns.

A fourth additional example, alone or in combination with one or more of the first through third additional examples, process1600may include using data stored in the right set of memory elements at a conclusion of a most recently completed row-CPU mode computing process as input data to the respective column-accelerator mode rows.

A fifth additional example, alone or in combination with one or more of the first through fourth additional examples, process1600further includes determining if any neural network layers of the two-dimensional systolic array have any intermediate data transfer across cores remaining to be eliminated, and while intermediate data transfer across cores remains to be eliminated, continuing cycling through operations of the process1600in a circular sequence from row-CPU mode operations, to column-accelerator mode operations, to column-CPU mode operations, and to row-accelerator mode operations (block1630).

AlthoughFIG.16shows example blocks of process1600, in some implementations, process1600may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted inFIG.16. Additionally, or alternatively, two or more of the blocks of process1600may be performed in parallel.

The functions, acts or tasks illustrated in the Figures or described may be executed in a digital and/or analog domain and in response to one or more sets of logic or instructions stored in or on non-transitory computer readable medium or media or memory. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, microcode and the like, operating alone or in combination. The memory may comprise a single device or multiple devices that may be disposed on one or more dedicated memory devices or disposed on a processor or other similar device. When functions, steps, etc. are said to be “responsive to” or occur “in response to” another function or step, etc., the functions or steps necessarily occur as a result of another function or step, etc. It is not sufficient that a function or act merely follow or occur subsequent to another. The term “substantially” or “about” encompasses a range that is largely (anywhere a range within or a discrete number within a range of ninety-five percent and one-hundred and five percent), but not necessarily wholly, that which is specified. It encompasses all but an insignificant amount.

Other systems, methods, features and advantages will be, or will become, apparent to one with skill in the art upon examination of the figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the disclosure, and be protected by the following claims.