LeNet‐5 improvement based on FPGA acceleration

The Journal of EngineeringVolume 2020, Issue 13 p. 526-528 The 3rd Asian Conference on Artificial Intelligence Technology (ACAIT 2019)Open Access LeNet-5 improvement based on FPGA acceleration Yong Hou, Corresponding Author Yong Hou aspnetcs88@163.com School of Computer Engineering in Bengbu College, Bengbu, People's Republic of ChinaSearch for more papers by this authorZhang Bao Chen, Zhang Bao Chen School of Electronics and Electrical Engineering in Bengbu College, Bengbu, People's Republic of ChinaSearch for more papers by this author Yong Hou, Corresponding Author Yong Hou aspnetcs88@163.com School of Computer Engineering in Bengbu College, Bengbu, People's Republic of ChinaSearch for more papers by this authorZhang Bao Chen, Zhang Bao Chen School of Electronics and Electrical Engineering in Bengbu College, Bengbu, People's Republic of ChinaSearch for more papers by this author First published: 23 July 2020 https://doi.org/10.1049/joe.2019.1190Citations: 2AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Deep learning is a branch of traditional machine learning. Thanks to the leap in computer hardware computing power in recent years, deep learning has become one of the hot topics. In view of the inefficiency of the traditional LeNet-5, here, LeNet-5 is improved by using the FPGA to accelerate the LeNet-5. Finally, by training the handwritten digit recognition model experiment, it is verified that the improved LeNet-5 model has a great improvement in terms of efficiency and accuracy. 1 Introduction Digital recognition has been applied to all aspects of life, such as parking lot by license plate number, traffic electronic eye capture violations, large-scale data statistics, and electronic file storage. The method of handwritten digit recognition can also be applied to other character recognition problems, such as the recognition of English letters. Early researchers have achieved good results in the direction of digital recognition [1], such as using K-nearest classification method, support vector machines (SVM) classification method, Boosting classification method and so on. However, there are some shortcomings in these methods. For example, the K-proximity method needs to load all the training data sets into the memory during prediction, and then use the digital image to be tested and the training set as the sum of the corresponding pixel spreads, and the final smallest difference value is the predicted result. Obviously, such a method is not reliable in normal picture accuracy, and the requirements for handwritten digits to be tested are also high. The model with the best recognition rate should be convolutional neural networks (CNN) based on deep learning. The most typical example is LeNet-5, which was first used in the USA to handwritten numbers to identify bank checks [2]. It can be seen that handwritten digit recognition based on deep learning is quite reliable in terms of accuracy. In this paper, on the ZYNQ7 processor [3], the standard LeNet-5 is accelerated by an OpenCL-based field programmable gate array (FPGA) [4]. The experiments have shown that the improvements made to LeNet-5 have a good effect. 2 Improvement of LeNet-5 2.1 LeNet-5 model The LeNet-5 is a convolutional neural network designed by Yann LeCun for handwritten digit recognition in 1998. Most banks in the USA used it to identify handwritten numbers on cheques. It is the most representative experiment in early convolutional neural networks. The LeNet-5 model has seven layers. Fig. 1 shows the architecture of the LeNet-5 model. Fig. 1Open in figure viewerPowerPoint Basic structure of LeNet-5 The structure of each layer of the LeNet-5 model is described in detail below. The first layer is the convolution layer. The input of this layer is the original image pixel, and the input layer size accepted by the LeNet-5 model is 32 × 32 × 1. The first convolutional layer filter has a size of 5 × 5 and the depth is 6, which is not filled with all 0 s, with a step size is one. Since no all-zero padding is used, the output of this layer has a size of 32 − 5 + 1 = 28 and the depth is 6. The total number of parameters in this convolutional layer are 5 × 5 × 1 × 6 + 6 = 156, of which 6 are bias term parameters. Since the node matrix of the next layer has 28 × 28 × 6 = 4704 nodes, and each node is connected with 5 × 5 = 25 current layer nodes, the total number of connections in this layer has 4704 × (25 + 1) = 122304. The second layer is the pooling layer. The input of this layer is the output of the first layer, which is a 28 × 28 × 6 node matrix. The filter size used in this layer is 2 × 2, and the length and width are both 2, so the output matrix size of this layer is 14 × 14 × 6. The third layer is the convolution layer. The input matrix size of this layer is 14 × 14 × 6, and the filter size used is 5 × 5 and the depth is 16. This layer does not use all-zero padding, the step size is 1, and the output matrix size of this layer is 10 × 10 × 16. According to the standard convolutional layer, this layer should have 5 × 5 × 6 × 16 + 16 = 2416 parameters and 10 × 10 × 16 × (25 + l) = 41600 connections. The fourth layer is the pooling layer. The input matrix size of this layer is 10 × 10 × 16, and the filter size is 2 × 2, and the step size is 2. The output matrix size of this layer is 5 × 5 × 16. The fifth layer is a fully connected layer. The input matrix size of this layer is 5 × 5 × 16, and the number of output nodes of this layer is 120, and the total number of parameters is 5 × 5 × 16 × 120 + 120 = 48120. The sixth layer is a fully connected layer. The number of input nodes in this layer is 120, and the number of output nodes is 84. The total parameters are 120 × 84 + 84 = 10164. The seventh layer is a fully connected layer. The number of input nodes in this layer is 84, and the number of output nodes is 10. The total parameters are 84 × 10 + 10 = 850. 2.2 Implementation of the convolutional layer The top view of the convolutional layer is shown in Fig. 2. The convolutional layer is mainly composed of control logic, shared multiplier block, adder tree, and ReLU processing [5]. The weights of the feature map and the convolution kernel are stored in the on-chip buffer. The control logic scans the line according to the timing, gives the corresponding input feature read address and weight read address, and the write address of the output buffer; The read data given by the feature buffer and the weight buffer are processed by the multiplier block, and then entered into the adder tree to complete the offset addition and ReLU operations, and then write to the output feature buffer. Fig. 2Open in figure viewerPowerPoint Top view of the convolutional layer 2.3 Implementation of the pooling layer The top view of the pooled layer is shown in Fig. 3. The structure of the pooling layer is similar to that of the convolutional layer, except that it has its own control module and computing unit. The control module generates a memory read and write address to scan the entire input feature map. As of the maximum pooling, the calculation unit is a comparator. The operations of different input layers are parallel. Fig. 3Open in figure viewerPowerPoint Top view of the pooling layer 2.4 Implementation of full layer The full layer and convolution layer are very similar, the difference is that the number of weights is large, not a single convolution kernel, so the control module generates some different addresses. In addition, the output result of the full layer needs to be used at the same time in the next layer, so it is stored in the register without being stored in the RAM. 2.5 LeNet-5 accelerator The top view of the ZYNQ-based LeNet-5 accelerator system is shown in Fig. 4. The system includes a ZYNQ7 processor for transmitting the input picture accelerator (the weight and offset parameters have been made into coe and imported into the ram/rom IP). The direct memory access (DMA) engine [6] is used to implement data transfer from off-chip DDR3 [7] memory to FPGA slices. These standard IPs are all connected to the advanced eXtensible interface (AXI) bus [8]. When the picture is imported into DDR3, the DMA engine starts to transfer data to the on-chip storage and starts the operation of the CNN accelerator. After the operation ends, the accelerator notifies the processor through the PL interrupt, and the processor obtains the operation result. Fig. 4Open in figure viewerPowerPoint Top view of the LeNet-5 accelerator Convolutional neural networks typically interpret low-level input features as advanced features through the cooperation of multiple convolutional and pooled layers. These advanced features can be categorised by a limited number of class outputs through the final classification layer, such as the fully connected layer. The convolutional layer is to perform a three-dimensional multiplication and multiply-accumulate operation with N_if input feature maps and N_K K × K-sized convolution filters (kernels), and finally obtain an output characteristic map. The calculation formula is shown in (1), and the calculation process is shown in Fig. 5. (1) where and represent the neural unit values at the point (x, y) of the input feature map f _i and the output feature map , respectively. is the value of the kernel at the point () used to convolve with the input feature map fi to produce the output feature map . The pooling layer is usually placed behind the convolutional layer to reduce the dimensions of the input feature map. The algorithm replaces the input unit of size with the maximum value or the average value. As shown in Fig. 5, the full layer is the final classification layer. In this layer, the input feature and the fully connected weight matrix are matrix multiplied to calculate the final output feature. Convolution and full joins usually end up with a non-linear activation function, such as tanh, sigmoid, and ReLU. ReLU is now very popular because it converges quickly in training and is easy to implement in hardware. The formula for ReLU is f = max(0,x). Fig. 5Open in figure viewerPowerPoint LeNet-5 convolution and pooling process 3 Experiment In order to verify the effectiveness of the proposed algorithm, the current mainstream LeNet-5 improved algorithm [9-11] is compared with the LeNet-5 improved algorithm proposed by this paper. The experimental platform is ubuntu (64bit) operating system, i7 quad-core 8-thread CPU, clocked at 4 GHz, 30 TB mechanical hard disk, 500 GB solid state hard disk, 256 GB memory, ZYNQ processor, and FPGA development board. The software environment is python, TensorFlow, and quartus. In the experiment, the data flow of the improved lennet-5 is roughly as follows: The initial input image (32 × 32) is stored in the position f 1 _ram where the input image of the first layer is stored. After the first layer is convolved, six images (28 × 28) are obtained, which are stored in the position f 2 _ram where the input image of the second layer is stored. After the second layer is pooled, six images (14 × 14) are obtained, which are stored in the position f 3 _ram where the input image of the third layer is stored; after the third layer is convolved, 16 images (10 × 10) are obtained, which are stored in the four-layer input image is stored in the location f 4 _ram; after the fourth layer is pooled, 16 images (5 × 5) are obtained, which are stored in the location f 5 _ram where the input image of the fifth layer is stored; the fifth, sixth, and seventh layers are fully connected layer, the results are stored directly in the register. The output of the last layer is 10 nodes, and the maximum output category and value are selected after the comparator. Table 1 shows the recognition rate of different LeNet-5 improved algorithm structures after 40 iterations using the test set. The recognition rate of LeNet5 improved in this paper is 99.62%, the recognition rate of LeNet5 in literature [9] is 96.05%, the recognition rate of LeNet5 in literature [10] is 97.49%, and the recognition rate of LeNet5 in literature [11] is 95.19%. The training time is 10.5, 24.8, 53.1 and 42.9 min, respectively. Table 1. Recognition rate of the different improved LeNet-5 after training Name iLeNet-5 LeNet5 [9] LeNet5 [10] LeNet5 [11] Rr 99.62% 96.05% 97.49% 95.19% Tt (min) 10.5 24.8 53.1 42.9 iLeNet-5 in Table 1 is referred to improved LeNet-5, which is proposed in this paper. Rr is referred to recognition rate in Table 1. Tt is referred to training time in Table 1. LeNet5 [9] is referred to the LeNet5 proposed in literature [9]. LeNet5 [10] is referred to the LeNet5 proposed in literature [10]. LeNet5 [11] is referred to the LeNet5 proposed in literature [11]. Fig. 6 are the curves comparing the different LeNet5 algorithms and the improved LeNet-5 algorithm proposed in this paper, in terms of recognition accuracy. Through the analysis and comparison of the line graph, it can be seen that the LeNet-5 improved in this paper converges rapidly after 30 iterations. The average accuracy of training and testing tends to 0.99, and the cross-entropy loss tends to 0.02. The LeNet-5 of the literature [9, 10] has to increase the recognition rate by 206 iterations. LeNet-5 in [11] has increased the number of link weights caused by the addition of full connections, and it takes 238 iterations to greatly improve the accuracy of recognition. By comparison, it can be seen that all LeNet-5 algorithms [9-11] used for comparison can converge within 432 iterations in general, and the average accuracy of training and testing tends to be 0.86, cross-entropy loss. It tends to 0.04. Therefore, the LeNet-5 algorithm improved in this paper has the fastest convergence rate and the highest accuracy, and the classification effect is obviously better than the other LeNet-5 algorithms used for comparison. Fig. 6Open in figure viewerPowerPoint Accuracy line chart for different LeNet-5 algorithms 4 Conclusion LeNet-5 has been improved in this article. The experimental results show that the improved LeNet-5 algorithm has obvious advantages in terms of test accuracy and training time compared to the improved LeNet-5 algorithm in [9-11]. The effectiveness of LeNet-5 improved in this paper is demonstrated. In the next step, the LeNet-5 will continue to be improved for further boost the accuracy of digital classification and try to apply it to more classification fields. 5 Acknowledgments This work is supported by Science Research Project of Bengbu College (project number: 2017ZR02zd), 2018 outstanding talents training projects in Anhui Province (Research and Application of Multi-Instance Learning Algorithms (project number: gxyq2018107)) and 2017 Bengbu College-level quality engineering projects (Big Data Technology and Application (project number: 2017ghjc8)). 6 References 1Jin S., Di S., Liang X. et al.: 'DeepSZ: A novel framework to compress deep neural networks by using error-bounded lossy compression'. HPDC'19: The 28th Int. ACM Symp. on High-Performance Parallel and Distributed Computing, Phoenix, AZ, USA, 2019, pp. 159 – 170 2Martin C.H., Mahoney M.W.: ' Traditional and heavy-tailed self regularization in neural network models', 2019, pp. 26 – 37 3Belov A.A., Klimov P.A., Sharakin S.A.: 'The network architecture of the data-processing system for the photodetector of an orbital detector of ultra-high energy cosmic rays', Instrum. Exp. Tech., 2018, 61, (1), pp. 27 – 33 4Zhang J., Li J.: 'Improving the performance of OpenCL-based FPGA accelerator for convolutional neural network'. Acm/sigda Int. Symp. on Field-programmable Gate Arrays, Monterey, CA, USA, 2017, pp. 25 – 34 5Li Y., Yuan Y.: ' Convergence analysis of two-layer neural networks with ReLU activation', 2017, pp. 65 – 70 6Tan J., Yang Y., Hadri H.E. et al.: 'Fast quantification of nanorod geometry by DMA-spICP-MS', Analyst, 2019, 144, (7), pp. 2275 – 2283 7Gong Y., Yu F.: 'Design of high-speed real-time sensor image processing based on FPGA and DDR3'. IEEE Int. Conf. on Computer & Communications, Chengdu, People's Republic of China, 2018, pp. 2105 – 2109 8Xin L., Cai Y., Han J. et al.: 'A high utilization FPGA-based accelerator for variable-scale convolutional neural network'. 2017 IEEE 12th Int. Conf. on ASIC (ASICON), Guiyang, People's Republic of China, 2017, pp. 944 – 947 9Deak N., Creţ O., Hedeşiu H.: ' Efficient FPGA floorplanning for partial reconfiguration-based applications', 2019, pp. 309 – 309 10Tao Z.: ' Level-1 track finding with an all-FPGA system at CMS for the HL-LHC', 2019, pp. 155 – 164 11Rajagopal K., Nazarimehr F., Guessas L. et al.: 'Analysis, control and FPGA implementation of a fractional order modified shinriki circuit', J. Circuits Syst. Comput., 2019, 28, (14), p. 1950232 Citing Literature Volume2020, Issue13July 2020Pages 526-528 FiguresReferencesRelatedInformation