# An Energy-Efficient Convolutional Neural Network Processor Architecture Based on a Systolic Array
## 3 System Modules  
### PE and PU  
The processing element (PE) is the basic computing unit of the processor and is used to complete the multiplication and addition (MA) operations of convolution and fully connected computation. The schematic structure of the PE is shown in Figure 8a, which has two operation functions: (1) Multiply--Add: I \(\times\) W + B = Psum; (2) Multiply--Accumulate: I \(\times\) W + Psumi-1 = Psumi. The first function realizes the process of adding the product of the image value and weight value with the bias value transferred from outside the PE. The second function realizes the process of accumulating the product generated by the current computation with the partial sum produced by the last computation. The Sel0 and Clear signals of PE are used to switch the operation function and clear the partial sum register to zero, respectively.  
Among the latest popular CNNs, the filter with a size of \(3\times 3\) is the most widely used. Therefore, the proposed CNN processor is mainly aimed at CNNs with filter size = 3. According to the previously mentioned computing strategy, each PE completes the 1D convolutions of one filter row, so three PEs are needed to form the systolic array, and two FIFOs are added to temporarily store the intermediate values output by PE0 and PE1, which constitutes the basic processing unit (PU) for 2D convolution, as shown in Figure 9. During the convolution process, SEL2 is configured to 0 to select the input source of the bias buffer as the output of PE2, the image buffer broadcasts image data to all PEs in the PU, each filter buffer outputs one filter row to the corresponding PE, and the bias buffer outputs the bias values to PE0. After computing starts, the PEs are enabled in the order of PE0-PE1-PE2 until all PEs are enabled, which process the 1D convolutions simultaneously, realizing the parallel operation between filter rows. The result of the current channel output from PE2 is stored in the bias buffer as the bias values for the next channel operation. After the convolution of all channels is completed, all PEs are disabled and the bias buffer stores the output feature map.  
Figure 10 shows the process of the PU performing fully connected computation. We can see that only PE0 participates in the computation. At this time, SEL2 is configured as 1, and the output of PE0 is selected as the input source of the bias buffer. The final result will be directly stored in the bias buffer without being temporarily stored in FIFO. The design of PE0 is slightly different from other PEs because it has some logic and signals related to fully connected computation. Such a PE is called Big PE, and its circuit structure is shown in Figure 8b. The Sel1 signal of the PE will be configured as 1 to directly obtain weight data from the AXI-Stream interface for computation.  
Figure 8: (**a**) Schematic of the PE; (**b**) schematic of the Big PE.  
Figure 9: Diagram of the PU performing the convolution operation.  
The PE array in the proposed processor consists of 128 PUs, including 384 PEs. Such a design is an optimal compromise between the hardware resources available in the Xilinx ZC706 platform (AMD, Santa Clara, CA, USA) used in this paper and the throughput rate of the processor. This PE array is capable of performing parallel computation among 128 channels of the output feature map. Limited by a maximum bandwidth of 1024 bit of AXI-Stream on the ZC706 platform, the PE array is able to process the fully connected computation of 64 output neurons in parallel when the weight values are 16-bit fixed point. Of course, if we use a platform with more bandwidth and storage resources, the PE array can be scaled up to obtain a higher throughput.