Revisiting Deep Learning Parallelism: FineGrained Inference Engine Utilizing Online Arithmetic In this paper, we revisit the parallelism of neural inference engines. In a departure from the conventional coarsegrained neuronlevel parallelism, we propose a synapselevel parallelism by performing highly parallel finegrained neural computations. Our method employs online Most Significant Digit First (MSDF) digitserial arithmetic to enable early termination of the computation. Using online MSDF bitserial arithmetic for DNN inference (1) enables early termination of ineffectual computations, (2) enables mixedprecision operations (3) allows higher frequencies without compromising latency, and (4) alleviates the infamous weights memory bottleneck. The proposed technique is efficiently implemented on FPGAs due to their concurrent finegrained nature, and the availability of onchip distributed SRAM blocks. Compared to other bitserial methods, our FineGrained Inference Engine (FGIE) improves energy efficiency by xc3x971.8 while having similar performance gains.

Towards efficient deep neural network training by FPGAbased batchlevel parallelism Training deep neural networks (DNNs) requires a significant amount of time and resources to obtain acceptable results, which severely limits its deployment in resourcelimited platforms. This paper proposes DarkFPGA, a novel customizable framework to efficiently accelerate the entire DNN training on a single FPGA platform. First, we explore batchlevel parallelism to enable efficient FPGAbased DNN training. Second, we devise a novel hardware architecture optimised by a batchoriented data pattern and tiling techniques to effectively exploit parallelism. Moreover, an analytical model is developed to determine the optimal design parameters for the DarkFPGA accelerator with respect to a specific network specification and FPGA resource constraints. Our results show that the accelerator is able to perform about 10 times faster than CPU training and about a third of the energy consumption than GPU training using 8bit integers for training VGGlike networks on the CIFAR dataset for the Maxeler MAX5 platform.