# A Review of the Optimal Design of Neural Networks Based on FPGA
## 5 Design of the DNN Accelerator and Acceleration Framework Based on FPGA
### FPGA Accelerator Design for Different Neural Networks
#### 5.1.3 The GAN Accelerator Based on FPGA  
GANs mainly consist of a generator and a discriminator, producing better output through mutual game learning between generator G and discriminator D. In the original GAN theory, both G and D are not required to be neural networks, as long as the corresponding functions that are generated and discriminated against can be fitted. However, in practical application, deep neural networks used as G and D generally, so in the GAN accelerator design based on FPGA, it is basically aimed at the generator and the discriminator to optimize. Therefore, it is essentially optimized for CNN, RNN, or other neural network, mainly from the lower network number, wherein the accelerators are designed from the perspectives of reducing storage pressure, reducing computational complexity, and improving network computing speed. Reference [69] focused on optimizing multiplicative and cumulative operation in GANs to reduce power consumption. In some works, the binarized GAN [131] and the ternary GAN [132] were used to reduce the precision of weight data, so as to improve the operation speed. In [133], the deconvolution operation in GAN was optimized to achieve higher throughput and resource utilization. On the basis of optimizing deconvolution operation, the authors of [134] used the pruning operation to reduce the computational complexity and the scale of the network model. Some scholars used a new fast transformation algorithm (FTA) for deconvolution calculation, which solved the computational imbalance problem well and eliminated the extra memory requirement of overlapping parts and sums [135]. In addition, the authors of [136] optimized the intensive convolution computation in GAN and the invalid computation caused by the insertion of a \"zero\" value in the deconvolution operation, using the reconfigurable mechanism to switch the two convolution functions flexibly on the same process element group (PEG), thus increasing the utilization of on-chip resources and improving parallelism. In order to solve the problem of high computational complexity and the need to store a large amount of intermediate data in the training of GAN on an embedded platform, a reconfigurable accelerator based on FPGA was also proposed in [137] for effective GAN training. The acceleration framework is shown in Figure 16. Firstly, the cascade fast FIR (finite impulse response) algorithm (CFFA) is optimized for GAN training, and the fast convolution processing element (FPCE) based on the optimization algorithm is introduced to support various computing modes during GAN training. In the input prefetcher module and weight prefetcher module, 16-bit fixed point and 8-bit fixed point were used, respectively, to process data, greatly improving the data processing speed. Finally, the architecture achieved 315.18 GOPS performance and 83.87 GOPS/W energy efficiency on the Xilinx VCU108 FPGA platform with 200 MHz operating frequency. Experiments showed that the architecture was able to achieve high energy efficiency with less FPGA resources. Although the architecture can effectively avoid the large consumption of hardware resources by cascading small parallel FIR structures to larger parallel FIR structures, this operation greatly increases the delay. When the number of cascades reaches a certain limit, the performance loss caused by the delay is incalculable. Table 2 shows the performance comparison of different neural networks deployed with different optimization methods on different FPGA platforms in recent years. From Table 2, we can see that the use of Winograd and other convolution optimization technologies can bring about great performance gains because the most important operation in the neural network is the convolution operation, which occupies most of the operation of the whole neural network and consumes the largest number of resources and has the longest processing time. Therefore, the optimization for a convolution operation can bring about much higher performance gains than other optimization techniques.