## 3 Method
### Hardware Accelerator Structure Design of Convolution Layer  
As mentioned above, the LeNet-5 network has five network layers, mainly including three convolution layers and two fully connected layers. Therefore, the structure of these two network layers can be optimized when designing a neural network accelerator. The first three network layers are all convolution layers with the same implementation methods. In this design, the convolution layer, activation function and pooling layer were placed in the same module, which reduces the difficulty of interface design and improves the readability and maintainability of the code. The data flow operation process of hardware accelerator of convolution layer is divided into the convolution operation, ReLU activation function and pooling operation. The convolution operation is expressed as:  
\[f_{i}^{out}=\sum_{j=1}^{in}f_{j}^{in}\otimes w_{i,j}+b_{i} \tag{1}\]  
where \(w_{i,j}\) is the convolution kernel corresponding to the \(j\)-th input feature graph and the \(i\)-th output feature graph, and \(b_{i}\) is the bias of the \(i\)-th output feature graph. The activation function ReLU is expressed as:  
\[f(x)=max(x,0) \tag{2}\]  
where \(x\) is the value of the input feature graph, and \(f(x)\) is the value of the output feature graph. The pooling operation is expressed as:  
\[f_{i,j}^{out}=\underset{p*p}{max}\left( \begin{array}{ccc}f_{m,n}^{in}&\cdots& f_{m,n+p-1}^{in}\\ \vdots&&\vdots\\ f_{m+p-1,n}^{in}&\cdots&f_{m+p-1,n+p-1}^{in}\end{array} \right) \tag{3}\]  
where \(p\) is the sampling core size of maximum pooling. The redundant information in the feature map can be eliminated by the pooling layer operation, which can reduce the amount of computation, but will not affect the accuracy of recognition. Through analysis of Equations (1)-(3), the hardware accelerator operation process of the convolution layer is expressed as:  
\[f_{conv\ layer}^{out}=MaxPool(ReLU(Conv(x,w_{kernel},b))) \tag{4}\]  
The convolutional layer operation model represented by Equation (4) was mapped to the most basic hardware accelerator structure, as shown in Figure 5. By optimizing the basic structure, the overall processing performance of the convolution layer hardware accelerator was further improved. When a software model is transformed into a hardware structure by the HLS tool, the expression of the software model may adversely affect the final hardware accelerator structure. Typically, the convolution operations given in Equation (1) are usually implemented in nested loops. In order to minimize the impact of high latency caused by repeated reading of feature graphs, the sequence of convolution operations is described as follows (Algorithm 1). ```
OUT: Output IN: Input W: Weight R: Row of output feature map C: Column of output feature map K: Kernel size CHin: In-Channel size CHout: Out-Channel size 1. for(kr = 0; kr < K; kr++)
2 for(kc = 0; kc < K; kc++)
3 for(r = 0; r < R; r++)
4 for(c = 0; c < C; c++)
5 for(chi = 0; chi < Chin; chi++)
6 for(cho = 0; cho < CHout; cho++)
7. OUT[chol[r][c] = W[cho][chi][kr][kc] * IN[chi][r + kr][c + kc];
```  
**Algorithm 1**Convolution  
In the implementation of the hardware accelerator, the Pipeline command can be used to shorten the instruction trigger interval within the for cycle, so as to improve throughput and reduce delay. As shown in Table 2, in the absence of the Pipeline command, both the latency and interval for completing an image recognition are 7,709,158 clock cycles. After the Pipeline command was added in the fifth line of code of the algorithm given above, the latency and interval of one image recognition significantly reduced from 7,709,158 to 353,263 clock cycles, and the performance was about 22 times improved. However, resource utilization has also increased significantly. For example, the use of digital signal processing block DSP48E increased by about 27 times, and the use of flip-flop (FF) and Look-Up Table (LUT) has also increased by about 2 times.