Integer ConvNets on Embedded CPUs: Tools and Performance Assessment on the CortexA Cores Quantization via fixedpoint representation is commonly used to reduce the complexity of Convolutional Neural Networks (ConvNets). It is particularly suited for accelerating edgeinference on embedded devices as it enables to reduce resource requirements with no loss of prediction quality. However, porting integer ConvNets on lowend CPUs is not straightforward: it calls for proper software design and organization with a high degree of hardware awareness. Today there are plenty of fixedpoint libraries integrated into different inference engines which provide design support. The aim of this work is to review the most stable tools and analyze their performance on different usecases processed on embedded boards powered by Arm CortexA cores. The collected results provide an interesting analysis with useful guidelines for developers and hardware designers.

Static Block FloatingPoint Quantization for Convolutional Neural Networks on FPGA Convolutional neural networks (CNNs) have been widely applied in various computer vision and speech processing applications. However, the algorithmic complexity of CNNs hinders their deployment in embedded systems with limited memory and computational resources. This paper proposes static block floatingpoint (BFP) quantization, an effective approach involving KullbackLeibler divergence, to determine the static shared exponents. Without need for retraining, the proposed approach is able to quantize CNNs to 8 bits with negligible accuracy loss. An FPGAbased hardware design with static BFP quantization is also proposed. Compared with 8bit integer linear quantization, our experiments show that the hardware kernel based on static BFP quantization can achieve over 50% reduction in logic resources on an FPGA. Based on static BFP quantization, a tool implemented in the PyTorch framework is developed, which can automatically generate optimised configuration according to user requirements for given CNN models, where the entire optimization process takes only a few minutes on an Intel Xeon Silver 4110 CPU.