# A Review of the Optimal Design of Neural Networks Based on FPGA
## 5 Design of the DNN Accelerator and Acceleration Framework Based on FPGA
### Accelerator Design for Specific Application Scenarios
#### 5.2.1 FPGA Accelerator for Speech Recognition  
Speech recognition is a technology that enables machines to automatically recognize and understand human spoken language through speech signal processing and pattern recognition. In short, it is the technology that allows a machine to transform a speech signal into a corresponding text or command through the process of recognition and understanding. Speech recognition has been applied in many fields, including speech recognition translation, voice paging and answering platforms, independent advertising platforms, and intelligent customer service. Speech recognition has strong real-time performance and high delay requirements, and thus it generally relies on the FPGA platform.  
In terms of accelerating the application of speech recognition, the authors of [143] focused on the preprocessing stage of speech signals and proposed the use of a GAN to enhance speech signals and reduce noise in speech signals, so as to improve speech quality and facilitate speech recognition and processing. In order to reduce energy consumption and improve the speed of speech recognition, the authors of [138] proposed an FPGA accelerator structure called balanced row dual-ratio sparsity inducing pruning algorithm (BRDS) for speech recognition, as shown in Figure 17. The accelerator compresses the LSTM network by pruning algorithm in order to reduce computational complexity and uses data reuse and pipeline design to achieve low power consumption and low delay. However, the acceleration architecture does not consider the problem of multi-core parallel load imbalance after LSTM model compression, which may occur in the actual use, thus affecting the performance of the whole accelerator.  
Figure 17: BRDS: FPGA accelerator for speech recognition.  
To solve this problem, as early as 2017, the ESE system architecture proposed in [58] can be well solved. The proposed hardware structure is shown in Figure 18. The architecture takes the ESE accelerator on FPGA as the core and solves the problem of multi-core parallel load imbalance by using multiple channel units composed of processing units and activation vector queue units. The processing unit adopts a double-buffer structure and ping-pong data transmission mode to improve the execution efficiency. The faster processing unit can obtain data from the activation vector queue unit and continue to work without waiting for other processing units. At the same time, the problem of workload imbalance between different processing units is solved. Compared with CPU (i7 5930 K) and GPU (GTX Titan X), the ESE architecture on the Xilinx XCKU060 FPGA platform was 43 times and 3 times faster than CPU and GPU platforms, respectively, and the energy efficiency ratio was 40 times and 11.5 times higher, respectively. However, the architecture design is relatively complex, involving an FPGA hardware accelerator, a CPU software program, and external memory modules. Task allocation and scheduling coordination among the three modules have become the bottleneck restricting the performance of the architecture [13].