PULPNN: A Computing Library for Quantized Neural Network inference at the edge on RISCV Based Parallel Ultra Low Power Clusters We present PULPNN, a multicore computing library for a parallel ultralowpower cluster of RISCV based processors. The library consists of a set of kernels for Quantized Neural Network (QNN) inference on edge devices, targeting byte and subbyte data types, down to INT1. Our software solution exploits the digital signal processing (DSP) extensions available in the PULP RISCV processors and the clusteru0027s parallelism, improving performance by up to 63xc3x97 with respect to a baseline implementation on a single RISCV core implementing the RV32IMC ISA. Using the PULPNN routines, the inference of a CIFAR10 QNN model runs in 30xc3x97 and 19.6xc3x97 less clock cycles than the current stateoftheart ARM CMSISNN library, running on an STM32L4 and an STM32H7 MCUs, respectively. By running the library kernels on the GAP8 processor at the maximum efficiency operating point, the energy efficiency on GAP8 is 14.1xc3x97 higher than STM32L4 and 39.5xc3x97 than STM32H7.

Integer ConvNets on Embedded CPUs: Tools and Performance Assessment on the CortexA Cores Quantization via fixedpoint representation is commonly used to reduce the complexity of Convolutional Neural Networks (ConvNets). It is particularly suited for accelerating edgeinference on embedded devices as it enables to reduce resource requirements with no loss of prediction quality. However, porting integer ConvNets on lowend CPUs is not straightforward: it calls for proper software design and organization with a high degree of hardware awareness. Today there are plenty of fixedpoint libraries integrated into different inference engines which provide design support. The aim of this work is to review the most stable tools and analyze their performance on different usecases processed on embedded boards powered by Arm CortexA cores. The collected results provide an interesting analysis with useful guidelines for developers and hardware designers.