FineGrained Exploitation of Mixed Precision for Faster CNN Training As deep convolutional neural networks (CNNs) have become increasingly popular and successful at an everwidening number of machine learning tasks specialized hardware has become increasingly available for training and deploying them. NVIDIAu0027s recent Volta architecture includes tensor cores which perform a fused operation reduced and mixed precision (16bit multiply, 32bit accumulate). Recent research indicates that, typically, very little is lost (in terms of training accuracy) when half precision is used in place of single precision, and performance gains can be made by doing arithmetic in reduced precision. In this work we demonstrate that making layerbylayer choices as to the arithmeticdata precision can lead to further performance improvement. In our study of 25,200 CNNs we demonstrate an average speedup (over purely half precision) of 1.27x and speedups as high as 3.64x by appropriately combining single and half precision arithmetic and data types on a layerbylayer basis.

Automatic Generation of MultiPrecision MultiArithmetic CNN Accelerators for FPGAs Modern deep Convolutional Neural Networks (CNNs) are computationally demanding, yet real applications often require high throughput and low latency. To help tackle these problems, we propose Tomato, a framework designed to automate the process of generating efficient CNN accelerators. The generated design is pipelined and each convolution layer uses different arithmetics at various precisions. Using Tomato, we showcase stateoftheart multiprecision multiarithmetic networks, including MobileNetV1, running on FPGAs. To our knowledge, this is the first multiprecision multiarithmetic autogeneration framework for CNNs. In software, Tomato finetunes pretrained networks to use a mixture of short powersof2 and fixedpoint weights with a minimal loss in classification accuracy. The finetuned parameters are combined with the templated hardware designs to automatically produce efficient inference circuits in FPGAs. We demonstrate how our approach significantly reduces model sizes and computation complexities, and permits us to pack a complete ImageNet network onto a single FPGA without accessing offchip memories for the first time. Furthermore, we show how Tomato produces implementations of networks with various sizes running on single or multiple FPGAs. To the best of our knowledge, our automatically generated accelerators outperform closest FPGAbased competitors by at least 24xc3x97 for lantency and throughput; the generated accelerator runs ImageNet classification at a rate of more than 3000 frames per second.