Low-power accelerators for cognitive computing

Deep Neural Networks (DNNs) have achieved tremendous success for cognitive applications, and are specially efficient in classification and decision making problems such as speech recognition or machine translation. Mobile and embedded devices increasingly rely on DNNs to understand the world. Smartphones, smartwatches and cars perform discriminative tasks, such as face or object recognition, on a daily basis. Despite the increasing popularity of DNNs, running them on mobile and embedded systems comes with several main challenges: delivering high accuracy and performance with a small memory and energy budget. Modern DNN models consist of billions of parameters requiring huge computational and memory resources and, hence, they cannot be directly deployed on low-power systems with limited resources. The objective of this thesis is to address these issues and propose novel solutions in order to design highly efficient custom accelerators for DNN-based cognitive computing systems. As a first step, we characterize popular DNNs for different applications on General-PurposeArchitectures (GPAs) such as CPU and GPGPU. DNNs are composed of multiple layers of neurons whose core operation is the dot product between the inputs of each neuron and its weights. The weights of a DNN model are learned during the training procedure and used at inference. We observe that the main performance and energy bottleneck of GPA systems is the memory hierarchy due to the huge number of inputs, weights, and partial products, resulting in numerous data movements. Thus, we develop a baseline architecture for accelerating DNN inference based on a state-of-the-art DNN accelerator. After identifying the memory subsystem as the main bottleneck, we introduce an on-chip eDRAM memory to store the weights, and an SRAM buffer to store inputs and temporal partial results. The on-chip memories favor the reuse of data among different executions of the DNN, reducing the accesses to main memory, and improving the performance and energy efficiency of the accelerator. Efficient support for DNNs on CPUs, GPUs and accelerators has become a prolific area of research, resulting in a plethora of techniques for energy-efficient DNN inference. However, previous proposals focus on a single execution of a DNN. Popular sequence processing applications, such as speech recognition or video classification, require many back-to-back executions of the same DNN to process a sequence of inputs (e.g., audio frames, images). In the first contribution of this thesis, we show that consecutive inputs exhibit a high degree of similarity, causing the inputs/outputs of the different layers to be extremely similar for successive frames of speech or images of a video. Based on the high degree of input similarity observed, we propose DISC, a hardware accelerator implementing a Differential Input Similarity Computation technique to reuse some results of the previous execution, instead of computing the entire DNN. Computations related to inputs with negligible changes can be avoided with minor impact on accuracy, saving a large percentage of computations and memory accesses. Our results show that, on average, more than 60% of the inputs of any neural network layer tested exhibit negligible changes with respect to the previous execution. Avoiding the memory accesses and computations for these inputs results in 63% energy savings on average. On the other hand, DNN quantization is a common optimization that reduces the precision of the inputs and weights, saving storage and simplifying the computations. Prior works exploit