Static Block FloatingPoint Quantization for Convolutional Neural Networks on FPGA Convolutional neural networks (CNNs) have been widely applied in various computer vision and speech processing applications. However, the algorithmic complexity of CNNs hinders their deployment in embedded systems with limited memory and computational resources. This paper proposes static block floatingpoint (BFP) quantization, an effective approach involving KullbackLeibler divergence, to determine the static shared exponents. Without need for retraining, the proposed approach is able to quantize CNNs to 8 bits with negligible accuracy loss. An FPGAbased hardware design with static BFP quantization is also proposed. Compared with 8bit integer linear quantization, our experiments show that the hardware kernel based on static BFP quantization can achieve over 50% reduction in logic resources on an FPGA. Based on static BFP quantization, a tool implemented in the PyTorch framework is developed, which can automatically generate optimised configuration according to user requirements for given CNN models, where the entire optimization process takes only a few minutes on an Intel Xeon Silver 4110 CPU.

A BlockFloatingPoint Arithmetic Based FPGA Accelerator for Convolutional Neural Networks Convolutional neural networks (CNNs) have been widely used in computer vision applications and achieved great success. However, largescale CNN models usually consume a lot of computing and memory resources, which makes it difficult for them to be deployed on embedded devices. An efficient blockfloatingpoint (BFP) arithmetic is proposed in this paper. compared with 32bit floatingpoint arithmetic, the memory and offchip bandwidth requirements during convolution are reduced by 50% and 72.37%, respectively. Due to the adoption of BFP arithmetic, the complex multiplication and addition operations of floatingpoint numbers can be replaced by the corresponding operations of fixedpoint numbers, which is more efficient on hardware. A CNN model can be deployed on our accelerator with no more than 0.14% top1 accuracy loss, and there is no need for retraining and finetuning. By employing a series of pingpong memory access schemes, 2dimensional propagate partial multiplyaccumulate (PPMAC) processors, and an optimized memory system, we implemented a CNN accelerator on Xilinx VC709 evaluation board. The accelerator achieves a performance of 665.54 GOPs and a power efficiency of 89.7 GOPsW under a 300 MHz working frequency, which outperforms previous FPGA based accelerators significantly.