Architectures for multicore neuromorphic systems are provided. In various embodiments, a neural network description is read. The neural network description describes a plurality of logical cores. A plurality of precedence relationships are determined among the plurality of logical cores. Based on the plurality of precedence relationships, a schedule is generated that assigns the plurality of logical cores to a plurality of physical cores at a plurality of time slices. Based on the schedule, the plurality of logical cores of the neural network description are executed on the plurality of physical cores.

BACKGROUND

Embodiments of the present invention relate to architectures for multicore neuromorphic systems, and more specifically, to area-efficient, reconfigurable, energy-efficient, speed-efficient neural network substrates.

BRIEF SUMMARY

According to various embodiments of the present disclosure, methods of and computer program products for configuring a neural network are provided. A neural network description is read. The neural network description describes a plurality of logical cores. A plurality of precedence relationships are determined among the plurality of logical cores. Based on the plurality of precedence relationships, a schedule is generated that assigns the plurality of logical cores to a plurality of physical cores at a plurality of time slices. Based on the schedule, the plurality of logical cores of the neural network description are executed on the plurality of physical cores

According to various embodiments of the present disclosure, methods of and computer program products for operating a neural network are provided. A neuromorphic core is reconfigured. A plurality of inputs to the neuromorphic core is received. Substantially concurrently to receiving the plurality of inputs, a first plurality of outputs of a neuromorphic core are computed at a first time slice. Substantially concurrently to the computing, a second plurality of outputs of the neuromorphic core is sent. The second plurality of outputs is generated at a second time slice. The second time slice precedes the first time slice.

According to various embodiments of the present disclosure, a neurosynaptic system is provided. A reconfigurable neuromorphic core comprising a plurality of axons and a plurality of neurons is provided. An axon buffer is coupled to the plurality of axons. An off core memory is coupled to the neuromorphic core. An inter-core network is coupled to the neuromorphic core. The reconfigurable neuromorphic is operable to read a plurality of configuration parameters from the off core memory. The reconfigurable neuromorphic core is operable to reconfigure according to the plurality of configuration parameters prior to processing the plurality of inputs. The reconfigurable neuromorphic is operable to receive a plurality of inputs. The reconfigurable neuromorphic core is operable to compute a first plurality of outputs at a first time slice substantially concurrently to receiving the plurality of inputs. The reconfigurable neuromorphic is operable to send outputs generated at a second time slice via the inter-core network substantially concurrently to the computing. The second time slice precedes the first time slice.

DETAILED DESCRIPTION

Arrays of extremely low power neurosynaptic processing units, called neurosynaptic cores, provide an architecture to solve exascale big data problems. These cores use spikes to encode information. In a network of neurosynaptic cores, neurons on each core can connect to any axon of any other neurosynaptic core (including itself). When a neuron spikes, it sends a spike packet that gets delivered to a target axon on a destination core.

In digital spiking neuromorphic systems, information is represented and delivered by spikes, where each spike is a digital packet of information, carrying one or more bits. For example, the IBM TrueNorth chip is a digital spiking neuromorphic system where each spike carries a single bit of information (a binary spike). Spiking neural networks such as TrueNorth are based on delivering packets of information over switched communication wires, thereby significantly reducing the required wiring. The presence of a spike is treated as receiving a 1, its absence represents a 0. More values can be coded into binary spikes using several different spike coding schemas.

A spike communication from a source neuron on a source core, to a target axon on a destination core, would effectively need to traverse certain number of hops via routers in a 2D grid in either the horizontal or vertical or a combination of both to be delivered to the target axon on a destination core. Each hop a spike packet traverses, consumes power and energy.

Within an exemplary neuromorphic system such as TrueNorth, a fixed amount of time is allowed for a spike to travel from its source neuron to its destination axon. This fixed window is referred to as a tick. The time a spike requires for its journey varies based on the distance the spike must travel and the number of 2-D mesh routing, chip and board interfaces that the spike travels across.

On each tick, the neurons in a core are processed sequentially, starting with the first neuron and continuing through the last neuron. Accordingly, in addition to the transmission delays discussed above, each spike is also delayed by some additional fixed amount based on which neuron on a core generated it. For example, in an exemplary neuromorphic system such as TrueNorth having 256 neurons per core, the 256th neuron is not processed until the preceding 255 neurons are processed.

According to various embodiments of the present disclosure, a neurosynaptic program represents a neurosynaptic network. A neurosynaptic program includes information relating to the neurosynaptic network. In some embodiments, the information includes neuronal properties and dynamics that determine an electronic neuron's response to input spikes. For example, neuronal properties and dynamics can include a threshold parameter, a leak parameter, a delay parameter, or a reset parameter. In some embodiments, the neurosynaptic program information includes synaptic connections of the neuron (e.g., synaptic connections made via a synaptic crossbar). In some embodiments, the neurosynaptic program information includes axon properties (e.g., axon types). In some embodiments, the neurosynaptic program information includes one or more destinations (e.g., target axons) that the neuron's output spike should be delivered to.

According to various embodiments, a neurosynaptic network represents an instantiation of a neurosynaptic program. A neurosynaptic network may be instantiated in hardware, in simulation or in both. For example, a neurosynaptic program may give rise to one or more instances of a neurosynaptic network, wherein the instances may reside on a single core, multiple cores, or multiple chips.

According to various embodiments, a neuromorphic core circuit represents an example neurosynaptic network described by a neurosynaptic program.

According to various embodiments, a corelet or a Corelet Programming Language represent software that provide abstraction of neurosynaptic programs. A composition of neurosynaptic programs may be created by composing corelets.

A TrueNorth program is a complete specification of a network of neurosynaptic cores, along with its external inputs and outputs. In various embodiments, a divide-and-conquer approach is adopted whereby a large network of neurosynaptic cores is constructed by interconnecting a set of smaller networks of neurosynaptic cores, where each of the smaller networks, in turn, could be constructed by interconnecting a set of even smaller networks, and so on, down to a network consisting of a single neurosynaptic core, which is the fundamental non-divisible building block. This programming paradigm is referred to as Corelet Programming.

It will be appreciated from the above exemplary description that neural networks are fundamentally parallel and distributed, thus giving inherent speed advantages that can be exploited in various embodiments. Likewise, neural networks are fundamentally energy-efficient due to the use of localized memory and event-driven computation. However, neural networks may require dedicated hardware per neuron. Requiring dedicated hardware is contrary to general CMOS methodologies, which leverage rewritability and reprogrammability.

Accordingly, the present disclosure allows preservation of the parallelism and energy efficiency advantages of a neural network while mitigating the limitations inherent in dedicated hardware within planar CMOS technology.

Brain-inspired (or neuromorphic) computing combines energy-efficiency and speed-efficiency with tileability. However, these advantages generally come at the expense of area efficiency, which is generally higher in conventional von Neumann architectures. Accordingly, in various embodiments the present disclosure provides for neuromorphic systems that maintain the energy advantages and parallelism of neuromorphic computing while optimizing the area. In particular, in various embodiments, area is saved by folding a logical network onto a physical network. In this way, repeated computation can be exploited at the cost of increased energy. In various embodiments, reconfigurable synapse weights, neuron parameters, neuron biases, and neuron destinations enable a folding process.

With reference now toFIG. 1, a neurosynaptic core according to embodiments of the present disclosure is depicted. In some embodiments, neurosynaptic core100includes axons101, represented as rows, dendrites102, represented as columns, synapses103, represented as row-column junctions, and neurons104that receive inputs from dendrites. In some embodiments, there are 256 axons, and 256 neurons. In such embodiments, there are 256×256=65,536 synapses. Information flows from axons101to the neurons104, modulated by the synapses103. In various embodiments, the synapses may be binary, and may be associated with synaptic weights.

In some embodiments a plurality of neurosynaptic cores are tiled on a chip. In an exemplary embodiments, a 64 by 64 grid of cores is tiled, yielding 4,096 cores, for a total of 1,048,576 neurons and 268,435,456 synapses. In such embodiments, neurons, synapses, and short-distance connectivity are implemented by the core circuit. Long-distance connectivity is logical. An exemplary embodiment is depicted inFIG. 2. Mesh router201provides communication between cores. Also on a given core, neuron to core202and core to axon203communication links are provided.

A logical core according to various embodiment may be defined as a node with a input (incoming) edges and n output (outgoing) edges. A logical core c carries out the computation according to Equation 1 and sends Ncto targets Tc. In Equation 1, N and B are n×1 vectors of output neurons and biases, A is an a×1 vector of input axons, W is an n×a weight matrix, and σcis a function.
Nc=σc(Wc×Ac+Bc)   Equation 1

A network N of logical cores according to various embodiment may be defined according to Equation 2, where C denotes a set of logical cores, E denotes a set of directed edges between logical cores, I is a subset of C that denotes input cores, and O is a subset of C that denotes output cores.
N=(I,C,E,O)   Equation 2

Consider the case that N is a directed acyclic graph (DAG). Let X denote the input to cores I and let Y denote the output of cores O. In epoch t, input Xtis presented to the network, and the network computes Ot. Epoch identifier t is not important from epoch to epoch.

Such a DAG can be used to establish precedence relationship between logical cores. In the above example, input cores in I have no precedent. The precedence relationship between cores is used to ensure that a logical core is scheduled for computation on a physical core only after all logical cores that send incoming edges to it are already scheduled.

Two logical cores c=(Wc, Ac, Bc, σc) and d=(Wd, Ad, Bd, σd) are said to be identical if Wc=Wd; Bc=Bd; σc=σd; and neither c is a precedent to d nor d is precedent to c. To perform further optimization of identical cores, each core in a system is assigned an identification number such that all identical cores receive the same identification number.

A physical core is capable of receiving parameters (Wc, Ac, Bc, σc) for a logical core c. Given these parameters, a physical core can emulate the logical core c to compute Ncand send them to Tc.

Suppose a physical core is already loaded with parameters (Wc, Ac, Bc, σc) for logical core c. Now, for efficiency reasons, it is desirable to reuse these parameters rather than re-receiving them. So, it is desirable to emulate identical cores on the same physical core.

Suppose that there are P physical cores in the system. Let us suppose that these cores operate in lockstep. All cores go through a computation step followed by a communication step followed by a preparation step. To ensure that all communication and preparation is done, there can be a communication barrier requiring O(log P) time where all cores acknowledge that they are done and all messages are delivered.

According to various embodiments, a schedule maps the set of logical cores in a system to physical cores along with a sequence number. Given precedence and identity relationships between logical cores, it is desirable to map identical logical cores to the same physical core for energy-efficiency and to map logical cores to physical cores so as to minimize total run-time for the network. In some embodiments, these two preferences may conflict and in some embodiments, one or the other is optimized.

In some embodiments, a scheduler is provided to create a schedule. According to various embodiments, the scheduler ensures that a for a given physical core and for a given sequence number, the core has all the necessary ingredients. In particular, the scheduler ensures that a given physical core is properly configured with (Wc, Ac, Bc, σc) and that Tcis available to receive Ncupon computation.

According to some embodiments, neuron biases and destinations are loaded on a regular schedule. In some embodiments, weights are loaded on an irregular schedule depending upon weight reuse. In some embodiments, neuron firings move from source physical cores to destination physical core after each computation step.

Referring now toFIG. 3, an exemplary logical to physical scheduling process is illustrated according to embodiments of the present disclosure. A plurality of physical cores301. . .304are connected by an on-chip inter-core network305. . .307. In some embodiments, physical cores301. . .304are neurosynaptic cores as described above, having axons and neurons connected by synapses. Each physical core is paired with off-core storage memory308. . .311containing reconfiguration information for the cores. In some embodiments, neighboring cores may share an off-core storage, while in some embodiment each core has a dedicated off-core storage. In this way, one logical core is emulated by a physical core per computation step.

At a given time step τ, physical core301performs computation320. Computation is carried out for every physical core in parallel. In some embodiments, all neurons within each core are also updated in parallel. In some embodiments, axons are processed sequentially within each core.

Substantially concurrently with computation step320, the local results of prior computation step τ−1 are communicated321via inter-core network322. For every neuron that fired in step τ−1 (the previous step), a message is prepared and sent.

Substantially concurrently with computation step320and communication step321, the results of prior computation step τ−2 are processed323. Such results may originate from other physical cores via inter-core network322, or may original at the local physical core via loopback interface. For every physical core, all incoming messages are processed. Processing may include updating local axon buffers of the local core with appropriate inputs, or storing the message in a larger memory for later retrieval.

Also as part of processing step323, the data necessary for configuring the local physical core for the next computation step, at τ+1, is fetched from off-core storage308.

In some embodiments, a synchronization barrier324is included. Barrier324ensures that steps320,321, and323have completed before advancing from τ to τ+1.

In embodiments where a given computation step320at τ can depend on results from τ−3 or before, the scheduling constraint may be relaxed by sequentializing steps320,321,323, above. In some embodiments, two empty computation steps may be provided between consecutive non-empty computation steps to accommodate prior results.

In some embodiments, inter-core messages may be structured messages. Such messages are dispatched upon neuron firing on a source core during computation step τ, as described above. In some embodiments, the structured message may include components [q, Δ, A, N], where q is a target core, Δ is a difference such that the message will be used in computation step τ+Δ, A is a target axon, and N is the neuron firing value. It will be appreciated that the bound on Δ will dictate the size of the largest network. In some embodiments, a relative address is used for the target core. In this way, the cores may be tileable on a chip.

Upon receipt of a message at a core, for example of the [q, Δ, A, N] form described above, the message is processed. In particular, the q value is used for initial routing to a destination core. Based on the Δ value, the message is scheduled. In some embodiments, is if a message cannot be scheduled locally to the core, for example because the input buffer is of limited size, the message may be stored in the off-core storage. If the message can be scheduled locally to the core, the target axon A is updated with the neuron filing value N. In case Δ indicates a future time window, the appropriate slot in the axon buffer is updated.

If the core has not yet been initialized, the configuration parameters are fetched from off-core storage. The biases, destinations, and synapse weights are either loaded or scheduled in future, depending upon the size of the buffer. In some embodiments, initialization also includes fetching any previous neuron firing messages received and stored off-core.

In some embodiments, each axon buffer in a given core includes a valid bit. If this bit is zero, the core does not compute anything within a given computation step. After a valid axon buffer is used, the valid bit is set to zero and the used axon buffer is cleared.

Referring now toFIG. 4, an exemplary layout of core buffers is illustrated according to embodiments of the present disclosure. It will be appreciated that the depicted buffer sizes are merely exemplary, and that various buffer sizes are suitable for use according to various embodiments of the present disclosure. A given core401includes axon buffers402, which receive values as described above from other cores or from off-core storage. In some embodiments, as described further above, axon buffer402may correspond to individual time steps, and thus support queuing of inputs for future time steps. As noted above, in some embodiments, inputs that are scheduled further in advance than the axon buffers can accommodate may sent to off-core storage for future retrieval.

Neuron bias buffer403receives neuron biases from the off-core storage to configure core401. As discussed above, the neuron bias buffer403may correspond to individual time steps, and thus support queuing of configuration parameters for future time steps. Synapse buffer404receives synapse configuration information from the off-core storage to configure core401. As discussed above, the synapse buffer404may correspond to individual time steps, and thus support queuing of configuration parameters for future time steps. It will be appreciated that in some embodiments, one or more of the buffers may be omitted, allowing direct writing to the core or requiring that a given schedule have certain immutable parameters. For example, in some embodiments, the synapse buffer404is omitted.

Neuron outputs are gathered in neuron membrane potential registers405. The destinations are loaded from off-core storage, and then send to destination cores accordingly.

In some exemplary embodiments, axon buffers provide a signed 4-bit value per axon. In some embodiments, neuron biases are signed 4-bit values per neuron. In some embodiments, synapses are given signed 1-bit values per synapse. In some embodiments, axon buffers are implemented as circular queues.

In an exemplary embodiments, off-core storage is implemented as a map from [q, Δ] to the associated axon buffer, neuron biases, neuron destinations, synapse weights, neuron function, and reuse bit. However, it will be appreciate that a variety of data structures are suitable for storing configuration information according to the present disclosure

It will be appreciated that networks according to embodiments of the present disclosure are not limited to convolution networks. However, in some embodiments, convolution networks have greater energy-efficiency and speed as a result of weight reuse. Networks according to various embodiments support feed forward as well as recurrent connectivity. It will be appreciated that the network size may be tuned to obtain a desired trade-off between energy for speed.

Referring now toFIG. 5, an exemplary process for configuring a neural network is illustrated according to embodiments of the present disclosure. At501, a neural network description is read. The neural network description describes a plurality of logical cores. In some embodiments, the neural network description comprises a model file describing a network and a placement file describing the relative placement of cores.

At502, a plurality of precedence relationships are determined among the plurality of logical cores. At503, based on the plurality of precedence relationships, a schedule is generated that assigns the plurality of logical cores to a plurality of physical cores at a plurality of time slices. Based on the schedule, the plurality of logical cores of the neural network description are executed on the plurality of physical cores