Patent Description:
Spiking neuron network (SNN) hardware has been demonstrated to be a promising alternative to traditional Von-Neumann architecture for processing real-world intelligent applications, such as to emulate intelligence and human recognition and cognitive activity. In an SNN, a digital representation of neurons interact with neighboring neurons over synapses connecting the neurons. When a postsynaptic neuron receives a weighted spike over a synapse from a firing presynaptic neuron, a membrane potential of the postsynaptic neuron is increased. When the membrane potential of a neuron reaches a threshold, then that neuron fires a spike across all the synapses in which it is a presynaptic neuron. The synapses have plasticity to adjust their gain/weight according to one or more adaptation rules. Some of the different rules to adjust the synapse weight are based on a timing differential of when the postsynaptic and presynaptic neurons for a synapse fire a spike. Some of the weight adjustment rules include Spike Timing Dependent Plasticity (STDP) and Spike Timing Dependent Delay Plasticity (STDDP). With these techniques, the weight of the synapse used to adjust the spike transmitted over the synapse will be increased if a presynaptic spike arrives before the postsynaptic spike is fired and decreased if presynaptic spike arrives after the postsynaptic spike fires. The change of the weight is determined by the duration between the arrival times of two spikes.

A computational SNN maintains three sets of electronic storage: (<NUM>) for neurons, to store each neuron's potential; (<NUM>) for synapses, including weight, delay and connections; and (<NUM>) for spikes, to keep track of the timing information of when spikes are fired at the neurons.

<CIT> relates to a neural network system that includes a memory array that maintains information for multiple neurosynaptic core modules. Each neurosynaptic core module comprises multiple neurons.

<CIT> relates to a system for representing, storing, and reconstructing an input signal using spiking neuron networks.

<NPL>, relates to a spiking network that can polychronize, that is, exhibit reproducible time-locked but not synchronous firing patterns with millisecond precision, as in synfire braids. The network consists of cortical spiking neurons with axonal conduction delays and spike-timing-dependent plasticity (STDP).

Described embodiments provide improved techniques for maintaining spike history information on the timing of spikes at the neurons for use in adjusting the synaptic weights in a computational neural network.

Embodiments are described by way of example, with reference to the accompanying drawings, which are not drawn to scale, in which like reference numerals refer to similar elements.

Described embodiments provide techniques to reduce the number of operations required to maintain timing information on when neurons fire or spike. In described embodiments, a spike history array provides an array of memory cells, where rows provide timing information for neurons in columns representing a last number of time slots in which the timing of a last occurring spike is indicated for one of the neurons. The cells in the column for the current timeslot are cleared and only updated in rows of firing neurons that fire in that current timeslot. The other columns in the firing neuron row other than the entry for the current timeslot are cleared to indicate that there is no firing. In this way, a neuron row is only updated when the neuron represented by the row fires a spike. This optimizes operations to keep track of neuron firing by only having to update one column and the row of the firing neurons, and to provide an improved technique for recording the timing information and adjusting the synaptic weights based on timing differences of when the neurons fire.

In the following description, numerous specific details such as logic implementations, opcodes, means to specify operands, resource partitioning/sharing/duplication implementations, types and interrelationships of system components, and logic partitioning/integration choices are set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention is only defined by the scope of the appended claims. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.

Certain embodiments relate to storage device electronic assemblies. Embodiments include both devices and methods for forming electronic assemblies.

<FIG> illustrates an embodiment of a spiking neural network computing environment comprising a computing system <NUM> having a processor <NUM> and a memory <NUM> including an operating system <NUM>, which manages computer hardware and software resources, provides common services for computer programs, and manages program execution and scheduling, and a spiking neural network (SNN) program <NUM> to perform neural network computations. The SNN program <NUM> maintains a spike history array <NUM> providing timing information on when neurons in the neural network fire a spike on connected synapses, a now pointer <NUM> comprising a circular shift register indicating a current timeslot at which the neural network is processed, synapse information <NUM> providing information on each synapse over which a presynaptic neuron and postsynaptic neuron communicate; and neuron membrane potential information <NUM> having the membrane potential information accumulated for the neurons in the neural network. The computer <NUM> further includes a storage <NUM> in which programs, such as the operating system <NUM> and SNN program <NUM> and information <NUM>, <NUM>, and <NUM>, may be stored and loaded into the memory <NUM>. The combination of the spike history array <NUM>, synapse information <NUM>, and neuron membrane potential information <NUM> provide a computational representation of the neural network and its interconnected neurons.

The processor may comprise a neuromorphic core or neuromorphic machine including the components of the neural network implementation, such as <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>.

The memory <NUM> may comprise one or more non-volatile and/or volatile memory devices, such as a Flash Memory, a non-volatile dual in-line memory module (NVDIMM), DIMM, Static Random Access Memory (SRAM), Dynamic Random Access Memories (DRAMs), as NAND dies of flash memory cells, ferroelectric random-access memory (FeTRAM), nanowire-based non-volatile memory, three-dimensional (3D) cross-point memory, phase change memory (PCM), memory that incorporates memristor technology, Magnetoresistive random-access memory (MRAM), Spin Transfer Torque (STT)-MRAM and other electrically erasable programmable read only memory (EEPROM) type devices.

In one embodiment, the memory <NUM> may comprise one or more SRAM memory devices or other non-volatile memory devices known in the art. In an alternative embodiment, the memory <NUM> may comprise a main memory, such as a DRAM, for storing programs, e.g., <NUM> and <NUM>, being executed, and a separate non-volatile memory, such as an SRAM or flash memory, storing the spike history array <NUM>, now pointer <NUM>, synapse information <NUM>, and neuron membrane potential information <NUM>.

The storage <NUM> may comprise a non-volatile storage or memory, such as NAND dies of flash memory cells, NVDIMM, SRAM, ferroelectric random-access memory (FeTRAM), nanowire-based non-volatile memory, three-dimensional (3D) cross-point memory, phase change memory (PCM), memory that incorporates memristor technology, Magnetoresistive random-access memory (MRAM), Spin Transfer Torque (STT)-MRAM, a single level cell (SLC) Flash memory and other electrically erasable programmable read only memory (EEPROM) type devices. The storage device <NUM> may also comprise a magnetic storage media, such as a hard disk drive etc..

In <FIG>, the SNN program <NUM> is implemented in a neuromorphic computing device comprising a computer system <NUM> having a processor <NUM> and operating under control of an operating system <NUM>. In alternative embodiments, the neuromorphic computing device implementing the SNN program <NUM> may comprise a hardware device, such as an application specific integrated circuit (ASIC) or other hardware device including hardware logic, such as a system on a chip (SOC) dedicated to the SNN program <NUM> operations. Still further, the neuromorphic computing device may comprise a combination of a computer system and dedicated hardware.

<FIG> illustrates an embodiment of the spike history array <NUM> and now pointer <NUM>. The spike history array <NUM> comprises an array of memory cells having a row for each of the neurons in the neural network and N columns for N different timeslots. Each row for a neuron, e.g., Neuron <NUM>, Neuron <NUM>, Neuron m, in the spike history array <NUM> indicates the time slot at which the neuron most recently fired, where each column of the spike history array <NUM> corresponds to a column in the now pointer <NUM> providing a relative timeslot to the current timeslot at which processing is occurring, represented by the current entry <NUM> in the now pointer <NUM>.

In certain embodiments, after a neural network clock, timing cycle or timeslot, if a neuron fired during that time cycle, the entry or column for the current time <NUM> in the now pointer <NUM> is set to indicate a current time, e.g., set to a "<NUM>" and all other entries are set to "<NUM>". The current column or current entry <NUM> in the now pointer <NUM> corresponds to a current column <NUM> in the spike history array <NUM> array. If a spiking neuron fires in a timeslot identified by the current entry <NUM> in the now pointer <NUM>, the corresponding current column <NUM> in the firing neuron row in the spike history array <NUM> array of memory cells is set to "<NUM>" to indicate firing at the current time, and all other entries for other timeslots in the rows for idle neurons not firing are set to "<NUM>". In this way, each neuron row of the array <NUM> indicates a most recent time the neuron fired in the last N timeslots. The other cells in that column are set to "<NUM>" or some other value to indicate no firing, so that firing events in other timeslots other than the current timeslot are not indicated when the neuron fires at the current timeslot.

In one embodiment, the now pointer <NUM> comprises a circular shift register, such that for each time cycle, or change to a next timeslot, the values for the first entry <NUM> through next to last entry <NUM> are shifted to the right and the value in the last entry <NUM> is shifted to the first entry <NUM>, where all values are simultaneously shifted. In an alternative embodiment, the values for the first entry <NUM> through the last entry <NUM> may shift in a different manner, such as to the left, etc..

An entry i in the now pointer <NUM> to the left of the current entry <NUM> indicates the (current entry minus ith) previous timeslot and an entry j in the now pointer <NUM> to the right of the current entry <NUM> indicates the (N-(j-current entry))th previous timeslot.

<FIG> provides a timing illustration of entries in the now pointer <NUM> to timeslots at different time slots <NUM> through N+<NUM> timeslots, where there are N entries in the now pointer <NUM>. For each time slot row representation, for Time Slot = <NUM>, <NUM>, N, N+<NUM>, N+<NUM>, there is a value for each entry in the now pointer <NUM>; an absolute time row <NUM> for the <NUM> through N+<NUM> time slots; and a relative time <NUM> for each entry in the last N time slots relative to the current timeslot. The information in the relative time <NUM> row may be used to determine a time difference between two entries in the now pointer <NUM> and two corresponding columns in the spike history array <NUM> to determine the time difference between when two neurons fired based on the difference between columns in the neuron rows having a "<NUM>", indicating the time slot in which the two neurons last fired.

<FIG> illustrates an embodiment of a synapse information <NUM>i instance maintained for each synapse in the synapse information <NUM>, and includes a presynaptic neuron <NUM> identifier; a postsynaptic neuron <NUM> identifier; a synapse identifier <NUM> of a synapse between the neurons <NUM> and <NUM>; and a synaptic weight <NUM> applied to the spike fired from the presynaptic neuron <NUM> onto the synapse <NUM> that reaches the postsynaptic neuron <NUM>. There may be multiple synapses connecting to one postsynaptic neuron, i.e., fanning into the postsynaptic neuron, and one presynaptic neuron may fire onto multiple synapses to transmit to different postsynaptic neurons, i.e., fanning out to the postsynaptic neurons. The synapse information instances <NUM>i for all the synapses provides the arrangement and mapping of the neurons in the neural network.

The synaptic weight <NUM> may be calculated using spike time dependent (STDP) plasticity learning based on timing differences between presynaptic and postsynaptic firing spikes. In certain embodiments, for a synapse, when a postsynaptic spike occurs in a specific time window after a presynaptic spike, the weight of this synapse may be increased. Likewise, if the postsynaptic spike occurs before the presynaptic spike, the weight of the synapse may be decreased. The strength of the weight change is a function of time between presynaptic and postsynaptic spike events. Another algorithm for calculating the synapse weight is a Spike Timing Dependent Delay Plasticity (STDDP) algorithm. Both these synapse weighting algorithms of STDP and STDDP are based on timing differences of firings by the presynaptic <NUM> and postsynaptic <NUM> neurons. These techniques are used to determine the synaptic weight <NUM> to apply to a spike fired by the presynaptic neuron <NUM> on the synapse <NUM> that reaches the postsynaptic neuron <NUM>.

<FIG> illustrates an embodiment of a neuron membrane potential instance <NUM>i maintained for each neuron in the neural network in the membrane potential information <NUM>, and includes for an identified neuron <NUM> a membrane potential <NUM>. Additional information may also be provided, such as information used to determine the decay of a membrane potential. The membrane potential <NUM> for a neuron accumulates the potential values of weighted spikes received on the synapse for which the neuron <NUM> is a postsynaptic neuron. When the accumulated membrane potential <NUM> reaches a threshold, the neuron <NUM> may fire and the membrane potential <NUM> is then reset to zero. Further, a value in the membrane potential <NUM> may decay over time slots according to a decay algorithm for membrane potential.

<FIG>, <FIG>, and <FIG> illustrate an embodiment of operations performed by the spiking neural network (SNN) program <NUM> to process the spike history array <NUM> to determine timing differences for currently firing neurons used to adjust the synaptic weights <NUM>. With respect to <FIG>, upon a new clock cycle occurring (at block <NUM>) for the neural network operations, the SNN program <NUM> simultaneously shifts (at block <NUM>) the values in the first <NUM> through N-1th <NUM> entries in the now pointer <NUM> one cell to the right and shifts the value in the Nth entry <NUM> to the first entry <NUM>. The SNN program <NUM> determines (at block <NUM>) a location of the entry (LocX) in the now pointer <NUM>, or entry number, of the current timeslot <NUM>, having a "<NUM>". All the entries in the current column <NUM> of the spike history array <NUM> corresponding to the now pointer current entry <NUM> indicating the current timeslot are set (at block <NUM>) to zero to indicate no spike firing in the last N timeslots.

The SNN program <NUM> determines (at block <NUM>) all the firing neurons (FNi), if any, comprising those neurons <NUM> having a membrane potential <NUM>, in their neuron membrane potential information <NUM>i, greater than a threshold used to determine when to trigger a spike. If (at block <NUM>) there are no firing neurons <NUM>, i.e., only idle neurons, then control ends. If (at block <NUM>) there are one or more firing neurons (FNi), then to update the timing information in the spike history array <NUM>, the SNN program <NUM> writes (at block <NUM>) the N entries of the adjusted now pointer <NUM> across the N columns of the rows of the firing neurons in the spike history array <NUM>. Writing the now pointer <NUM> across the columns of one of the neuron rows, writes all zeros to the columns of the row that are not the current column <NUM> and writes a one ("<NUM>") to the current column <NUM> entry in the firing neuron row indicating that firing occurred in the current timeslot. Thus, writing of the now pointer <NUM> resets all the cells in the firing neuron rows to "<NUM>" that are not in current column <NUM>. The now pointer <NUM> may be simultaneously applied across all the firing neuron rows of the spike history array <NUM>.

With respect to <FIG>, a loop of operations is performed at block <NUM> through <NUM> for each of the determined firing neurons (FNi), comprising those neurons having a membrane potential <NUM> greater than a threshold.

At block <NUM>, a determination is made of the postsynaptic fan-out neurons <NUM> receiving the spike from the firing neuron <NUM> over synapses <NUM> (Si,j) whose presynaptic neuron <NUM> is the firing neuron (FNi). For each of the determined postsynaptic fan-out neurons <NUM>, the SNN program <NUM> updates (at block <NUM>) a potential <NUM>i of the postsynaptic fan out neuron based on the weight of the corresponding spike from the synaptic weight <NUM> and adds that weighted potential from the synapse to the membrane potential <NUM> for the determined postsynaptic fan-out neuron <NUM>.

Control then proceeds to block <NUM> in <FIG> where the SNN program <NUM> determines (at block <NUM>) from the synapse information <NUM> one or more connected neurons. A "connected neuron" comprises presynaptic fan-in neurons <NUM> (PreNj) of synapses <NUM> (Sj,i) in synapse information <NUM>, , instances for which the firing neuron (FNi) is the postsynaptic neuron <NUM> and postsynaptic fan-out neurons <NUM> (PostNk) of synapses <NUM> (Si,k) in synapse information <NUM>, , instances for which the firing neuron (FNi) is the presynaptic neuron <NUM> communicating the spike to the fan-out neurons. For each of the connected neurons, i.e., determined presynaptic fan-in neurons (PreNj) and postsynaptic fan-out neurons (PostNk), a loop of operations is performed at blocks <NUM> through <NUM>. If (at block <NUM>) there is no timing indicator, e.g., a "<NUM>", in the row for the connected neuron (PreNj or PostNk), then the synaptic weight <NUM> for the synapse <NUM> (Sj,i or Si,k) between the determined firing neuron (FNi) and the connected neuron (PreNj or PostNk) is not updated (at block <NUM>). If (at block <NUM>) there is a timing indicator, e.g., "<NUM>", indicating firing in the row of the determined presynaptic neuron (PreNj) in the last N time cycles, then the SNN program <NUM> determines (at block <NUM>) a location (LocY) of the entry in the row for the connected neuron (PreNj or PostNk) indicating last time fired, i.e., the number of the entry having a "<NUM>" value. The location LocY of the timing indicator, e.g., "<NUM>", for the connected neuron may be determined by using a one-hot decoder on the row of the array for the connected neuron to calculate the location or entry of the timing indicator, or "<NUM>".

The relative time difference for the firing time for the connected neuron (PreNj or PostNk) from the current time slot (LocX) depends on whether the column location of the presynaptic neuron (LocY) is to the right or left of the current time column <NUM>, <NUM>. If the LocY is to the left of the current column/time <NUM>, <NUM>, then the timing difference is LocX minus the LocY. If the LocY of the entry when the connected neuron (PreNj or PostNk) fired is to the right of the current time <NUM>, then the current time has wrapped, and the timing difference is LocX minus (N - (LocY - LocX)). In one embodiment, the time difference may be determined (at block <NUM>) according to equation (<NUM>) below, where N is the number of entries in the now pointer <NUM>, or number of columns in the spike history array <NUM>: <MAT>.

The SNN program <NUM> uses (at block <NUM>) the determined time difference to calculate an adjusted weight <NUM> for the synapse <NUM> and updates the synaptic weight <NUM> in the synapse information <NUM>i for the determined firing neuron as the presynaptic <NUM> or postsynaptic <NUM> neuron and the connected neuron as the postsynaptic <NUM> or presynaptic <NUM> neuron, respectively. The SNN program <NUM> may use the determined timing difference with an STDP or STDDP type algorithms, and other timing difference algorithms, to determine a synaptic weight <NUM> for the synapse to the firing neuron.

With the operations of <FIG>, <FIG>, and <FIG>, the SNN program <NUM> may update the spike history array <NUM> to indicate in the rows of the array for the firing neurons the timing at which the firing neurons fired a spike on a synapse and to update the synaptic weight <NUM> based on a timing differential for all synapses having one of the firing neurons as a postsynaptic neuron. With the described embodiments, the SNN program <NUM> minimizes the operations performed on the memory cells of the array <NUM> by first resetting all the bits in the current column <NUM> of the array <NUM> to zero, such as by setting them to ground, and then writing the row pointer across all the rows of the firing neurons, which may be performed simultaneously on all the word lines through the rows of the firing neurons in the memory array <NUM>.

<FIG> illustrates an embodiment of the memory <NUM> in which the spike history array <NUM> is implemented in an SRAM array <NUM> having write ports of write bit lines BL <NUM><NUM>, <NUM><NUM>. <NUM>N and BL' <NUM><NUM>, <NUM><NUM>. <NUM>N for each column of cells and column-wise RESET ports comprising two bit lines, BL <NUM><NUM>, <NUM><NUM>. <NUM>N and BL' <NUM><NUM>, <NUM><NUM>. <NUM>N for each column of cells. In each column, one reset bit line (BL) <NUM><NUM>, <NUM><NUM>. <NUM>N is connected to ground (GND) <NUM><NUM>, <NUM><NUM>. <NUM>N and the other bit line (BL') <NUM><NUM>, <NUM><NUM>. <NUM>N is connected to voltage (VCC) <NUM><NUM>, <NUM><NUM>. The array <NUM> includes word lines <NUM><NUM> (WL<NUM>), <NUM><NUM> (WL<NUM>), and <NUM>m (WLm) on the rows of the array <NUM>.

When Loci_RESET <NUM>i is "<NUM>", the NMOS gate and PMOS gate (<NUM>i & <NUM>i) are both closed, so the RESET bit lines BL <NUM>i and BL' <NUM>i are not connected to any power or ground resulting in nothing written into the column, i.e. cell value remains. When LOCi_RESET <NUM>i is "<NUM>", both gates are open, and BL <NUM>i is pulled down to ground and BL' <NUM>i is pulled up to VDD, so all cells along the current column are set to "<NUM>", i.e., the second value, as part of the operation at block <NUM> in <FIG>. After setting all the cells in the current column to "<NUM>" (second value) using the LOCi_RESET <NUM>i for the current column, the now pointer <NUM> value is asserted on the write bit lines (BL <NUM><NUM>, <NUM><NUM>. <NUM>N and inverted value on BL' <NUM><NUM>, <NUM><NUM>. <NUM>N) and then the WLi <NUM>i for the firing neurons will be activated, so the value on the bit lines can be written into the corresponding rows of cells, to perform the operation at block <NUM> in <FIG>.

In this way, there are two sets of bit lines <NUM>i/<NUM>i and <NUM>i/<NUM>i for each column, write bit lines <NUM>i/<NUM>i for normal write (spike information update) and another set of bit lines <NUM>i/<NUM>i for RESET purposes.

Since, the write data of the now pointer <NUM> is the same for all firing neurons that fire in that time slot, in one embodiment multiple of the word lines for the firing neurons can be activated at the same time by having multiple WL decoders with their outputs drive the WL lines simultaneously. In another embodiment, there may be one word line (WL) decoder and the word line (WL) driver signal for the firing neurons is OR-ed with an M-bit now pointer <NUM> selecting signal.

With the embodiment of the array <NUM> of <FIG>, at step <NUM> in <FIG>, the setting of the entries in the current column of the array to zero may be performed by resetting the cells in the current column <NUM> to ground, such as by using a LocX_RESET, on the entry X of the current column. The operation at block <NUM> in <FIG> to write the entries of the row pointer <NUM> to the rows of the firing neurons may be performed by asserting the now pointer <NUM> on all the word lines <NUM>i (WLi) for the rows of the firing neurons in the array <NUM>.

The described operations of the SNN program <NUM> may be implemented as a method, apparatus, device, computer product comprising a computer readable storage medium using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The described operations may be implemented as code or logic maintained in a "computer readable storage medium". The term "code" as used herein refers to software program code, hardware logic, firmware, microcode, etc. The computer readable storage medium, as that term is used herein, includes a tangible element, including at least one of electronic circuitry, storage materials, inorganic materials, organic materials, biological materials, a casing, a housing, a coating, and hardware. A computer readable storage medium may comprise, but is not limited to, a magnetic storage medium (e.g., hard disk drives, floppy disks, tape, etc.), optical storage (CD-ROMs, DVDs, optical disks, etc.), volatile and non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, Flash Memory, firmware, programmable logic, etc.), Solid State Devices (SSD), computer encoded and readable punch cards, etc. The computer readable storage medium may further comprise a hardware device implementing firmware, microcode, etc., such as in an integrated circuit chip, a programmable logic device, a Programmable Gate Array (PGA), field-programmable gate array (FPGA), Application Specific Integrated Circuit (ASIC), etc. Still further, the code implementing the described operations may be implemented in "transmission signals", where transmission signals may propagate through space or through a transmission media, such as an optical fiber, copper wire, etc. The transmission signals in which the code or logic is encoded may further comprise a wireless signal, satellite transmission, radio waves, infrared signals, Bluetooth, etc. The program code embedded on a computer readable storage medium may be transmitted as transmission signals from a transmitting station or computer to a receiving station or computer. A computer readable storage medium is not comprised solely of transmission signals, but includes physical and tangible components. Those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the present invention, and that the article of manufacture may comprise suitable information bearing medium known in the art.

In this way, the SNN program <NUM> may be implemented in a neuromorphic computing device comprising a computer readable storage medium or a hardware device, such as an application specific integrated circuit (ASIC), system on a chip (SOC), or a combination thereof.

<FIG> illustrates an embodiment of a computer architecture <NUM>, such as the components included in the computer system <NUM>, including a processor <NUM> that communicates over a bus <NUM> with a volatile memory device <NUM> in which programs, operands and parameters being executed are cached, and a non-volatile storage device <NUM>, such as target system memory <NUM>. The bus <NUM> may comprise multiple buses. Further, the bus <NUM> may comprise a multi-agent bus or not be a multi-agent bus, and instead provide point-to-point connections according to PCIe architecture. The processor <NUM> may also communicate with Input/output (I/O) devices 812a, 812b, which may comprise input devices, display devices, graphics cards, ports, network interfaces, etc..

In certain embodiments, the computer node architecture <NUM> may comprise a personal computer, server, mobile device or embedded compute device. In a silicon-on-chip (SOC) implementation, the architecture <NUM> may be implemented in an integrated circuit die.

The reference characters used herein, such as i, j, are used to denote a variable number of instances of an element, which may represent the same or different values, and may represent the same or different value when used with different or the same elements in different described instances.

The terms "an embodiment", "embodiment", "embodiments", "the embodiment", "the embodiments", "one or more embodiments", "some embodiments", and "one embodiment" mean "one or more (but not all) embodiments of the present invention(s)" unless expressly specified otherwise.

The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless expressly specified otherwise.

The enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise.

The terms "a", "an" and "the" mean "one or more", unless expressly specified otherwise.

Devices that are in communication with each other need not be in continuous communication with each other, unless expressly specified otherwise. In addition, devices that are in communication with each other may communicate directly or indirectly through one or more intermediaries.

A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

When a single device or article is described herein, it will be readily apparent that more than one device/article (whether or not they cooperate) may be used in place of a single device/article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device/article may be used in place of the more than one device or article or a different number of devices/articles may be used instead of the shown number of devices or programs.

Claim 1:
A neuromorphic computing device (<NUM>) for implementing a spiking neural network program (<NUM>) to maintain timing information on when neurons of the spiking neural network fire, the neuromorphic computing device comprising a processor (<NUM>) and a memory device (<NUM>) comprising an array (<NUM>, <NUM>) of rows and columns of memory cells, wherein there is one row of the rows for each of a plurality of neurons of the spiking neural network, and columns for each of a plurality of time slots, the neuromorphic computing device (<NUM>) configured to:
generate a spike history (<NUM>) in the memory device (<NUM>), wherein in response to each of at least one firing neuron of the neurons firing a spike over a synapse at a current timeslot, the memory device is to indicate, with a first value, in the current column in the row of the memory cells for the firing neuron that the spike was fired in the current timeslot and is to indicate, with a second value different to the first value, in the current column in rows of memory cells of idle neurons comprising neurons that did not fire that a spike was not fired;
use information in the array to determine a timing difference between a connected neuron and the firing neuron to which the connected neuron connects over a synapse; and
use the determined timing difference to adjust a weight of the synapse connecting the firing neuron and the connected neurons, wherein the determining of the timing difference comprises determining a difference of a first column location number in the array having the first value for the firing neuron and a second column location number in the array having the first value for the connected neuron and comprises determining a timing difference between the firing neuron and the connected neuron based on the first and second column location numbers.