# An Energy-Efficient Convolutional Neural Network Processor Architecture Based on a Systolic Array
## 2 System Architecture
### Systolic Array-Based Computing Strategy  
The systolic array represents a network of processing elements (PEs) that compute and transmit data rhythmically. These processing elements regularly input and output data to maintain regular dataflow [22]. The characteristics of the systolic array are modularization and regularization, which are very important in VLSI design. Figure 1: Proposed CNN processor architecture. All PEs in the systolic array are the same and work pipelined. Convolution can be completed in a systolic array. Taking a convolution operation with image size = 5 and filter size = 3 as an example, as shown in Figure 2, where i_row0-i_row4, w_row0-w_row2, b_row0-b_row2, and p_row0-p_row2 represent each row of the image values, weight values, bias values and output partial sum (psum) values, respectively, and taking p_row0 as an example, the computing process can be decomposed into:  
\[ \begin{split}\text{psum00}=\text{i\_row0}*\text{w\_row0}+\text{b\_row0},\\ \text{psum01}=\text{i\_row1}*\text{w\_row1}+\text{psum00},\\ \text{p\_row0}=\text{psum02}=\text{i\_row2}*\text{w\_row2}+\text{psum01},\end{split}  \tag{1}\]  
where psum00(01, 02) represents the psum produced by the one-dimensional (1D) convolution of i_row0(1, 2) and w_row0(1, 2), \(*\) is a convolution sign, it can be concluded that p_row1 and p_row2 have a similar computing process. These computing operations can be completed by a systolic array composed of three PEs, as shown in Figure 3. Each PE has multiplication and addition functions that can complete 1D convolution. Figure 4 shows the dataflow of a PE performing 1D convolution. The output results of each PE (such as psum00) will be temporarily stored in a FIFO as the bias values of the next PE computing. In the computing process, each row of image values is broadcast to all PEs simultaneously, and each PE completes the 1D convolution corresponding to one weight row, for example, the PE0 completes the 1D convolution of w_row0. The systolic array processing sequence is shown in Figure 5, where each dark circle represents the PE performing one 1D convolution using the corresponding image rows, weight rows and the transferred bias rows (psum rows) in unit time. The whole operation is carried out along the direction of the oblique upward arrow. Each PE obtains the psums of the previous PE from the FIFO as the bias values of its own computation. We can see from the figure that all PEs are enabled and disabled in the form of a systolic array: in 0, PE0 is enabled; in t1, PE1 is enabled; in t2, PE2 is enabled; in t3, PE0 is disabled; and in t4, PE1 is disabled; Finally, all PEs are disabled once all image rows are completed. Figure 3: Systolic array composed of three PEs. Figure 2: Example of a convolution operation. Table 1 shows the computation of all PEs and the results stored in FIFOs at each unit time, where psum10(11,12) represents the psum produced by the 1D convolution of i_row1(2, 3) and w_row0(1, 2), and psum20(21,22) represents the psum of i_row2(3, 4) and w_row0(1, 2). From this table we can see that the number of working PEs from t0 to t5 is 1, 2, 3, 2, 1. We can easily see that the utilization rate of the PEs in the systolic array will increase with the scale up of the input image, the majority of time all PEs are computing simultaneously in the case of a large image size. Only at the beginning and end of the convolution, the number of working PEs gradually increases from 0, then gradually decreases to 0, finally completing the convolution. The systolic array can also easily realize the convolution with multiple input feature map channels. Taking convolution with two channels as an example, the computing process of the first channel is the same as above. For the second channel, the convolution process is similar to that shown in Figure 2, whereby the image values and weight values are changed