A Fully Quantitative Scheme With Finegrained Tuning Method For Lightweight CNN Acceleration Based on the incremental network quantization (INQ) algorithm, this paper proposed a new quantization method with finegrained tuning (named FTINQ) to quantify convolutional weight of lightweight Convolutional Neural Network (CNN). The key idea of FTINQ is to employ extra 2bit tuning bits to provide more dense value space for the quantization process, so that the accuracy of floatingpoint CNN model can be well maintained. According to FTINQ, the floatingpoint multiplication can be simplified to the sum of two shifting operations. Compared with original INQ algorithm, testing results shows that FTINQ achieves 23% and 34% reduction on accuracy loss for MobilenetV1u0026V2, along with 2.5x improvements on training speed.

Static Block FloatingPoint Quantization for Convolutional Neural Networks on FPGA Convolutional neural networks (CNNs) have been widely applied in various computer vision and speech processing applications. However, the algorithmic complexity of CNNs hinders their deployment in embedded systems with limited memory and computational resources. This paper proposes static block floatingpoint (BFP) quantization, an effective approach involving KullbackLeibler divergence, to determine the static shared exponents. Without need for retraining, the proposed approach is able to quantize CNNs to 8 bits with negligible accuracy loss. An FPGAbased hardware design with static BFP quantization is also proposed. Compared with 8bit integer linear quantization, our experiments show that the hardware kernel based on static BFP quantization can achieve over 50% reduction in logic resources on an FPGA. Based on static BFP quantization, a tool implemented in the PyTorch framework is developed, which can automatically generate optimised configuration according to user requirements for given CNN models, where the entire optimization process takes only a few minutes on an Intel Xeon Silver 4110 CPU.