Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that i) approach state-of-the-art classification accuracy across 8 standard datasets, encompassing vision and speech, ii) perform inference while preserving the hardware's underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1200 and 2600 frames per second and using between 25 and 275 mW (effectively > 6000 frames / sec / W) and iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. For the first time, the algorithmic power of deep learning can be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer. . On the hardware front, building on foundational work on silicon neural systems [7] , neuromorphic computing, using novel architectural primitives, has recently demonstrated hardware capable of running 1 million neurons and 256 million synapses for extremely low power (just 70mW at real time operation) [8] . Bringing these approaches together holds the promise of a new generation of embedded, real-time systems, but first requires reconciling key differences in the structure and operation between contemporary algorithms and hardware. Here, we introduce and demonstrate an approach we call Eedn, Energy-efficient deep neuromorphic networks, which creates convolutional networks whose connections, neurons, and weights have been adapted to run inference tasks on neuromorphic hardware. For structure, typical convolutional networks place no constraints on filter sizes, whereas neuromorphic systems are constrained to block-wise connectivity that limits filter sizesthereby saving energy since weights can now be stored in local on-chip memory within dedicated neural cores. Here, we present a new convolutional network structure that naturally maps to the efficient connection primitives used in contemporary neuromorphic systems. We enforce this connectivity constraint by partitioning filters into multiple groups, and yet maintain network integration by interspersing layers whose filter support region, by using a small topographic size, is able to cover incoming features from many groups [9] . For operation, contemporary convolutional networks typically use high precision (≥ 32-bit) neurons and synapses to provide continuous derivatives and support small incremental changes to network state, both formally required for backpropagation-based gradient learning. In comparison, neuromorphic designs use 1-bit spikes to provide event-based computation and communication (consuming energy only when necessary) and use low-precision synapses to co-locate memory with computation (keeping data movement local and avoiding off-chip memory bottlenecks). Here, we demonstrate that by introducing two constraints into the learning rule -binaryvalued neurons with approximate derivatives and trinaryvalued ({−1, 0, 1}) synapses -it is possible to adapt backpropagation to create networks directly implementable using energy efficient neuromorphic dynamics. This draws inspiration from the spiking neurons and low-precision synapses of the brain [10] [18] . For input data, we employ a first layer to transform multi-valued, multi-channel input into binary channels using convolution filters that are learned via backpropagation [14] [18] and whose output can be sent on chip in the form of spikes. These binary channels, intuitively akin to independent components [19] learned with supervision, provide a parallel distributed representation to carry out high-fidelity, fault-tolerant computation without the need for high-precision representation. Critically, we demonstrate that bringing the above innovations together allows us to create networks that approach state-of-the-art accuracy performing inference on 8 standard datasets, running on a neuromorphic chip at between 1200 and 2600 frames per second (FPS), using between 25 and 275 mW. We further explore how our approach scales by simulating multi-chip configurations. Ease-of-use is achieved using training tools built from existing, optimized deep learning frameworks [20] , with learned parameters mapped to hardware using a high level deployment language [21]  Here, we provide a description of the relevant elements of deep convolutional networks and the TrueNorth neuromorphic chip, and describe how the essence of the former can be realized on the latter. Deep Convolutional Networks. A deep convolutional network is a multilayer feedforward neural network, whose input is typically image-like and whose layers are neurons that collectively perform a convolutional filtering of the input or a prior layer (Figure 1 ). Neurons within a layer are arranged in two spatial dimensions, corresponding to shifts in the convolution filter, and one feature dimension, corresponding to different filters.
The binary activation mode can use spiking response for computing and communication, which is an energy-efficient method because it consumes energy only when necessary REF .
Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
PNAS 113 (2016) 11441-11446
Computer Science Medicine