System for electronic learning synapse with spike-timing dependent plasticity using phase change memory

A system, method and computer program product for producing spike-dependent plasticity in an artificial synapse is disclosed. According to one embodiment, a method for producing spike-dependent plasticity in an artificial neuron comprises generating a pre-synaptic spiking event in a first neuron when a total integrated input to the first neuron exceeds a first predetermined threshold. A post-synaptic spiking event is generated in a second neuron when a total integrated input to the second neuron exceeds a second predetermined threshold. After the pre-synaptic spiking event, a first pulse is applied to a pre-synaptic node of a synapse having a phase change memory element. After the post-synaptic spiking event, a second varying pulse is applied to a post-synaptic node of the synapse, wherein current through the synapse is a function of the state of the second varying pulse at the time of the first pulse.

BACKGROUND

1. Field of the Invention

The present invention relates to artificial neural networks. In specific, the present invention relates to electronic learning synapses with spike-dependent plasticity using phase change memory.

2. Background of the Invention

The point of contact between an axon of a neuron and a dendrite on another neuron is called a synapse, and with respect to the synapse, the two neurons are respectively called pre-synaptic and post-synaptic. The essence of our individual experiences is stored in conductance of the synapses. The synaptic conductance changes with time as a function of the relative spike times of pre-synaptic and post-synaptic neurons, as per spike-timing dependent plasticity (STDP). The STDP rule increases the conductance of a synapse if its post-synaptic neuron fires after its pre-synaptic neuron fires, and decreases the conductance of a synapse if the order of the two firings is reversed. Furthermore, the change depends on the precise delay between the two events, such that the more the delay, the less the magnitude of change.

Artificial neural networks are computational systems that permit computers to essentially function in a manner analogous to that of biological brains. Artificial neural networks do not generally utilize the traditional digital model of manipulating 0s and 1s. Instead, neural networks create connections between processing elements, which are roughly functionally equivalent to neurons of a biological brain. Artificial neural networks may be comprised of various electronic circuits that are modeled on biological neurons.

BRIEF SUMMARY

According to one embodiment of the present invention, a method for producing spike-dependent plasticity in an artificial neuron comprises: generating a pre-synaptic spiking event in a first neuron when a total integrated input to the first neuron exceeds a first predetermined threshold; generating a post-synaptic spiking event in a second neuron when a total integrated input to the second neuron exceeds a second predetermined threshold; after the pre-synaptic spiking event, applying a first pulse to a pre-synaptic node of a synapse having a phase change memory element; and after the post-synaptic spiking event, applying a second varying pulse to a post-synaptic node of the synapse, wherein current through the synapse is a function of the state of the second varying pulse at the time of the first pulse.

According to another embodiment of the present invention, a method for producing spike-dependent plasticity in an artificial neuron comprises: generating a pre-synaptic spiking event in a first neuron; generating a post-synaptic spiking event in a second neuron; after the pre-synaptic spiking event, applying a first pulse to a gate of a transistor, the transistor connected in series between a first end and a ground of a phase change memory element in a synapse; and after the post-synaptic spiking event, applying a second varying pulse to a post-synaptic node of the synapse, wherein current through the synapse is a function of the state of the second varying pulse at the time of the first pulse, and wherein conductance of the phase change memory element depends on the current through the synapse.

According to another embodiment of the present invention, an apparatus for producing spike-dependent plasticity in an artificial neuron comprises: a phase change memory device connected at a first ed to a post-synaptic terminal; a transistor in series between a first end and a ground of the phase change memory device, a gate of the transistor being connected to a pre-synaptic terminal; and a first neuron and a second neuron, wherein the first neuron and the second neuron generate a plurality of voltage pulses to the pre-synaptic and post-synaptic terminals respectively, the first neuron and the second neuron receiving inputs from a plurality of neurons and generating pre-synaptic and post-synaptic internal spiking events respectively when a total integrated input to each neuron exceeds predetermined first and second thresholds, wherein in response to the pre-synaptic spiking event, the first neuron generates a first pulse at the pre-synaptic terminal that occurs a predetermined period of time after the pre-synaptic spiking event, and wherein in response to the post-synaptic spiking event, the second neuron generates a second pulse at the post-synaptic terminal that comprises a plurality of pulses of increasing frequency, such that current through the phase change memory device is a function of the state of the second pulse at the time of the first pulse.

According to another embodiment of the present invention, an apparatus for producing spike-dependent plasticity in an artificial neuron comprises: a synapse having post-synaptic and pre-synaptic terminals, and a phase change memory device, the phase change memory device connected at a first end to the post-synaptic terminal; a transistor connected to a second end of the phase change memory device, a gate of the transistor being connected to the pre-synaptic terminal; a first neuron connected to the pre-synaptic terminal generating a first pulse in response to a pre-synaptic spiking event; and a second neuron connected to the post-synaptic terminal generating a second pulse in response to a post-synaptic spiking event, wherein the phase change memory conductance depends on which one of the pre-synaptic or post-synaptic spiking events occur first.

According to another embodiment of the invention, a computer program product for producing spike-dependent plasticity in an artificial neuron, the computer program product comprises: a computer usable medium having computer usable program code embodied therewith, the computer usable program code comprising: computer usable program code configured to: generate a pre-synaptic spiking event in a first neuron when a total integrated input to the first neuron exceeds a first predetermined threshold; generate a post-synaptic spiking event in a second neuron when a total integrated input to the second neuron exceeds a second predetermined threshold; after the pre-synaptic spiking event, apply a first pulse to a pre-synaptic node of a synapse having a phase change memory element; and after the post-synaptic spiking event, apply a second varying pulse to a post-synaptic node of the synapse, wherein current through the synapse is a function of the state of the second varying pulse at the time of the first pulse.

DETAILED DESCRIPTION

Embodiments of the invention provide a system, method and computer readable medium for an electronic learning synapse with spike-timing dependent plasticity using phase change memory. The term “neuron” was coined by Heinrich Wilhelm Gottfried von Waldeyer-Hartz in 1891 to capture the discrete information processing units of the brain. The junctions between two neurons were termed “synapses” by Sir Charles Sherrington in 1897. Information flows only along one direction through a synapse, thus we talk about a “pre-synaptic” and a “post-synaptic” neuron. Neurons, when activated by sufficient input received via synapses, emit “spikes” that are delivered to those synapses that the neuron is pre-synaptic to. Neurons can be either “excitatory” or “inhibitory.”

A brain can be thought of as a directed graph where nodes are neurons and edges are synapses. The following table shows the rough number of neurons and synapses in a mouse, rat, and human. Each neuron in mammalian cortex makes roughly 8,000 synapses with other neurons.

The computation, communication, and memory resources of the brain all scale with the number of synapses and not with the number of neurons. Even power and space requirements scale with the number of synapses.

Some of the physical characteristics of the synapses are as follows. Synaptic density is roughly 7.2×108per mm3which roughly corresponds to placing synapses at three dimensional grids with 1 μm spacing in every direction. This figure seems to be a constant of nature across all mammalian cortices.

Synaptic weight is the influence that a pre-synaptic firing will have on post-synaptic neuron. Synaptic weights are plastic or adaptive, and change through time. Synaptic weight exhibits two forms of plasticity: (a) Long-term and (b) Short-term. Long-term changes in the transmission properties of synapses provide a physiological substrate for learning and memory, whereas short-term changes support a variety of computations. The mechanism of short-term plasticity is a form of gain control, and is not treated in this disclosure.

The mechanism of long-term weight adaptation is known as spike-timing dependent plasticity (STDP). Causality is a key element of STDP. Correlated activity can occur purely by chance, rather than reflecting a causal relationship that should be learned. Inputs that consistently are best at predicting a post-synaptic response should become the strongest inputs to the neuron. Thus in STDP, synapses are only strengthened if their pre-synaptic action potential precede, and thus could have contributed to, the firing of the post synaptic neuron. Accidental, non-causal coincidences will weaken synapses.

We describe one of the prevalent phenomenological description of STDP: (a) if pre-synaptic neuron fires t milliseconds before the post-synaptic neuron fires then the synaptic weight is increased (strengthened, potentiated) by A+exp(−t/τ) where A+and τ are constants; (b) if pre-synaptic neuron fires t milliseconds after the post-synaptic neuron fires then the synaptic weight is decreased (weakened, depressed) by A−exp(−t/λ) where A−and λ are constants.

If the synapse is assumed to be binary, then, at the broadest level, STDP can be summarized as follows: if the post-synaptic neuron fires within a short time of the pre-synaptic neuron, then synapse is turned fully ON, whereas if the pre-synaptic neuron fires within a short time of the post-synaptic neuron, then synapse is turned fully OFF. The STDP rule permits the brain to extract causality and correlations from a spatio-temporally varying environment.

A key characteristic of classical von Neumann computing is the separation of computation and memory. Specifically, if a memory location is to be modified, it is brought into a separate computing unit, modified, and then restored. This three-step process creates the classical von Neumann bottleneck that has plagued modern computer systems.

In contrast to von Neumann computing, synapses are memory elements that are modified in-place, that is, memory and computation are distributed in the brain. Embodiments of the present invention disclose a synapse-like device, which breaks the mold of traditional computing by creating a form of active memory. In general, embodiments of the present invention include a two-terminal device that exhibits synapse-like function, akin to STDP.

The present invention makes use of a unipolar memory-switching element whose program and erase operations can be accomplished with the same voltage polarity. An example of such a device is a phase-change memory (PCM). A phase-change memory device can be switched as follows: a lower voltage (current) pulse to program or set (that is, go from low conductance amorphous state to high conductance crystalline state) and a higher voltage (current) pulse to erase or reset (that is, go from high conductance to low conductance state). Embodiments of the invention make use of novel bipolar pre and post-synaptic pulses that can capture the essence of STDP in such materials. Specifically, by using the PCM device controlled by a MOS transistor, these novel pre and post-synaptic pulses can be shaped to program the device if post-synaptic pulse follows the pre-synaptic pulse within 100 ms, or erase the device if pre-synaptic pulse follows the post-synaptic pulse within 100 ms. The disclosed embodiments may also permit multiple conductance states, and can reward causality and punish anti-causality in a STDP-like way. In particular, embodiments of the invention match physical switching characteristics of the phase-change material to a workable approximation of the required STDP curve via a design of pulses that are applied to electrodes representing post-synaptic and pre-synaptic neurons.

FIG. 1shows a schematic of the artificial synapse system10which consists of an artificial binary synapse12, a first neuron15and a second neuron14. The artificial binary synapse12includes a two terminal PCM device16connected in series with a MOS transistor18. That is, one side of the MOS transistor18is connected to a first terminal of the PCM device16, and the other side of the MOS transistor18is connected to ground. The artificial binary synapse12includes a pre-synaptic node20and a post-synaptic node22. The first neuron15is connected to pre-synaptic node20through axon21. The second neuron14is connected to the post-synaptic node22through dendrite23. The gate of the MOS transistor18is connected to the pre-synaptic node20. A second terminal of the PCM device16is connected to the post-synaptic node22. The first neuron15has a dendrite,24and the second neuron14has an axon26. The first and second neurons are part of a network of neurons that connect to each other through synapses. These neurons may be processing elements that transmit information as electrical signals that propagate from the neuron cell body through the axon, past the synapse and through the dendrite. In this sense, the axon can be considered as the output terminal of the neuron, and the dendrite can be considered its input terminal.

The PCM device16is assumed to have a threshold voltage, VTh=0.9V. If the PCM device16is in the low conductance amorphous (RESET) state, voltages less than VThapplied across it will not cause current to flow, but voltages greater than VThwill result in threshold switching behavior, and current will flow through the device. Various PCM devices may be used in the embodiments of the invention. For example, these embodiments may use PCM cells fabricated in the mushroom cell configuration or the pore cell configuration; both configurations being known to those skilled in the art.

It is assumed that the application of 1.5V to the gate of the transistor, (the pre-synaptic node20) fully turns it ON, and the current through the PCM device16is then primarily determined by the magnitude of the voltage applied to the post-synaptic node22. In other words, when the transistor turns ON, the PCM device16becomes ready to conduct. However, whether the device conducts or not depends upon the precise voltage applied during this window at the post-synaptic node22.

To create an electronic synapse, the present invention addresses how the conductance of the PCM device16changes as a function of the relative timing between the onset of pre-synaptic and post-synaptic pulses. As discussed below, in accordance with embodiments of the invention, the resistance of the PCM device16can be manipulated by application of specific electric pulses.

FIG. 2shows a graph illustrating how the resistance of a PCM device may be increased starting from a low resistance state. By applying a current pulse to the device, with a magnitude in the range of 0.25 mA to 0.6 mA, lasting about 20-50 ns, and with a fall time less than 5 ns, the resistance of the PCM device can be increased from about 20 KΩ to about 20 MΩ. In the branch of the curve marked RESET, applying current pulses of increasing amplitude (shown in the X-axis) leads to higher resistance for the PCM cell.

FIG. 3shows a graph illustrating how to decrease the resistance of a PCM device starting from a high resistance state. By applying a sequence of current pulses to the device, with a constant magnitude in the range of 0.1 mA to 0.2_mA, lasting about 2-5 ns, the resistance of the PCM device can be decreased steadily from about 20 KΩ to about 20 MΩ.

In accordance with embodiments of the invention, the previously described characteristics of the PCM material can be utilized for the purpose of eliciting a synapse-like behavior by careful design of pulses that are applied to the pre-synaptic node20and post-synaptic nodes22shown inFIG. 1.

FIG. 4shows the pre-synaptic and post-synaptic pulses generated by the first and second neurons15,14in accordance with an embodiment of the invention. The neurons may comprise processing elements that include internal integrate and fire circuits, which are well known in the art. The purpose of these internal integrate and fire circuits is to continuously integrate the total input electrical quantity (which could be current, or charge) and to generate specific pulses that are applied at the axon21and dendrite23when that integrated input exceeds a predetermined threshold value. The exact value of the predetermined threshold may be determined based on various factors, such as the value of the current supplied by each input, the number of inputs, the desired performance of the neuron, and other factors.

The applied pre-synaptic pulse (labeled “Spre(t)”) may be a 1.5V spike of very narrow width (100-150 nsec), which is triggered 100 ms after the output of the internal integrate and fire circuit of the neuron exceeds a predetermined threshold value. The original spiking event of the first neuron15occurs at time t=0 in the graph inFIG. 4. The applied post-synaptic pulse is the waveform (labeled “Spost(t)”) and may be applied shortly after the output of the internal integrate and fire circuit of the second neuron14exceeds a predetermined threshold value. The original spiking event of the second neuron14also occurs at time t=0 in the graph inFIG. 4. The applied post synaptic pulse, may comprise a series of short pulses, each lasting 5 ns, arriving at a rate of about 40 MHz in the first 25 ns, 50 MHz between 25 to 50 ms, 66 MHz between 50 to 75 ms and 100 MHz pulses between 75 to 100 ms. After 100 ms, this pulse is a set of stair-case voltages, each stair lasting for 25 ms, and going down from 2V to 0.5V.

When there are no pre-synaptic and post-synaptic spikes, 0V is applied to both the pre-synaptic node20and the post-synaptic node22of the artificial synapse12shown inFIG. 1.

The main purpose of the transistor18is to restrict the current that could potentially flow through the synapse to just the time when the pre-synaptic signal is high. Thus, the energy dissipated per synaptic operation is greatly reduced, since current flows for 100-150 ns instead of ˜200 ms.

To further illustrate the various embodiments of the invention,FIGS. 5-7show the applied pre-and post-synaptic pulses in three different cases. In the first case, the original pre-synaptic neuron firing event30occurs just before the original post-synaptic neuron firing event32. In this embodiment, the time difference between the two events is about 20 ms. With this time difference, the overlap between the two pulses happens such that the resultant current through the PCM device16is a series of short pulses, lasting about 2 ns, which causes it to go to (or remain in) the SET state. It should be noted that the actual number of pulses that are applied may depend on the time difference between the original pre and post synaptic spikes. Larger number of pulses may be applied if there is a relatively small time delay between the spikes. The number of applied pulses may be decreased as this time delay increases. Since the PCM cell16resistance continuously decreases as a function of the number of pulses applied to it, the PCM cell16conductance is increased strongly for small time delays, and increased weakly as the time delay itself increases.

FIG. 6illustrates a second case, where the original pre-synaptic neuron firing event occurs just after the original post-synaptic neuron firing event, which is within 30 ms in this example. The result is that the current through the PCM cell16is a pulse of constant amplitude, where the amplitude is determined by the time delay between the arrivals of the original spikes. The large magnitude of current flowing through the PCM cell16causes it to go to (or remain in) the RESET state. For a relatively small time delay, there will be a large magnitude of current flowing through the cell, strongly RESETing the cell, or strongly decreasing the cell conductance. As the time delay between the pulses increases, the magnitude of current that flows through the cell also decreases, thereby decreasing the effectiveness of the RESET pulse.

FIG. 7shows a third case, where the original pre-synaptic spiking event occurs much before or after the original post-synaptic spiking event, that is, outside a desired maximum time window, which may be 105 ms in this embodiment. This results in a 0V pulse across the PCM cell16, because the transistor18is never turned on so the state of the PCM cell16does not change.

It is noted that the precise values of actual voltages are for illustration only and may vary with particular applications. It is the predetermined pulse shapes and timing control that are used by embodiments to achieve the advantages of the invention.

The above-discussed embodiments show how to create an artificial electronic synapse with multiple conductance states. However, if only a simpler binary synapse is desired then it can be achieved as follows. Specifically, in an additional embodiment of the invention, shown inFIG. 8, the same pre-synaptic pulse shown inFIG. 4may be used. However, the post-synaptic pulse as shown inFIG. 8may be simplified such that from 0 to 100 ms—a pulse train with uniform frequency is used—and from 100 to 200 ms a pulse with constant voltage level is used. This results in a binary synapse, with a single current level through the PCM16, instead of the plurality of current levels that depend on the relative timing of the original incoming spikes, as shown in the example inFIG. 6.

FIG. 9shows another alternative embodiment of the invention, which is the same asFIG. 8except that from 0-100 ms the series of voltage pulses in the post-synaptic pulse are replaced by a constant voltage followed by an increased voltage level that returns to zero at 200 ms. The magnitudes of the voltage levels for these two intervals are chosen such that if the overlap between the two signals happens between 0-100 ms, the resistance of the cell decreases, while an overlap between 100-200 ms leads to an increase in the resistance.

Anti-STDP has been widely observed, and is essentially the reverse of STDP. By simply flipping the post-synaptic pulse inFIG. 4around the y-axis at 100 ms—the resulting pulse can be used to elicit anti-STDP behavior.

FIG. 10shows a flowchart of a process30for causing an electronic learning synapse to exhibit spike-timing dependent plasticity using phase-change memory switching elements in accordance with an embodiment of the invention. At block32, first neuron15generates a pre-synaptic spiking event. At block34a second neuron14generates a post-synaptic spiking event. As described above, the order and relative timing of the pre-and post-synaptic spiking events may vary. A first period of time after the pre-synaptic spiking event, a first pulse is applied to a pre-synaptic node (such as node20) of a synapse (such as synapse12), having a phase change memory element, at block36. The phase change memory element may comprise phase change memory16. A second period of time after the post-synaptic spiking event, a second varying pulse may be applied to a post-synaptic node, such as node22, of the synapse, wherein the current through the synapse is a function of the state of the second varying pulse at the time of the first pulse. The shape of the first and second pulses may be, for example, as shown inFIG. 4, in one embodiment, or, as inFIGS. 8 and 9in other embodiments. In block40, the resistance of the PCM is modified as a function of the relative timing of the pre-and post-synaptic spiking events.

As can be seen from the above disclosure, embodiments of the invention provide an electronic learning synapse with spike-timing dependent plasticity using phase change memory-switching elements. As will be appreciated by one skilled in the art, the present invention may be embodied as a system, method or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.”

FIG. 11is a high level block diagram showing an information processing system useful for implementing one embodiment of the present invention. The computer system includes one or more processors, such as processor102. The processor102is connected to a communication infrastructure104(e.g., a communications bus, cross-over bar, or network). Various software embodiments are described in terms of this exemplary computer system. After reading this description, it will become apparent to a person of ordinary skill in the relevant art(s) how to implement the invention using other computer systems and/or computer architectures.

The computer system can include a display interface106that forwards graphics, text, and other data from the communication infrastructure104(or from a frame buffer not shown) for display on a display unit108. The computer system also includes a main memory110, preferably random access memory (RAM), and may also include a secondary memory112. The secondary memory112may include, for example, a hard disk drive114and/or a removable storage drive116, representing, for example, a floppy disk drive, a magnetic tape drive, or an optical disk drive. The removable storage drive116reads from and/or writes to a removable storage unit118in a manner well known to those having ordinary skill in the art. Removable storage unit118represents, for example, a floppy disk, a compact disc, a magnetic tape, or an optical disk, etc. which is read by and written to by removable storage drive116. As will be appreciated, the removable storage unit118includes a computer readable medium having stored therein computer software and/or data.

In alternative embodiments, the secondary memory112may include other similar means for allowing computer programs or other instructions to be loaded into the computer system. Such means may include a removable storage unit120and an interface122. Other examples of such means may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM, or PROM) and associated socket, and other removable storage units120and interfaces122which allow software and data to be transferred from the removable storage unit120to the computer system.

The computer system may also include a communications interface124. Communications interface124allows software and data to be transferred between the computer system and external devices. Examples of communications interface124may include a modem, a network interface (such as an Ethernet card), a communications port, or a PCMCIA slot and card, etc. Software and data transferred via communications interface124are in the form of signals which may be, for example, electronic, electromagnetic, optical, or other signals capable of being received by communications interface124. These signals are provided to communications interface124via a communications path (i.e., channel)126. This communications path126carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and/or other communications channels.

In this document, the terms “computer program medium,” “computer usable medium,” and “computer readable medium” are used to generally refer to media such as main memory110and secondary memory112, removable storage drive116, and a hard disk installed in hard disk drive114.

Computer programs (also called computer control logic) are stored in main memory110and/or secondary memory112. Computer programs may also be received via communications interface124. Such computer programs, when executed, enable the computer system to perform the features of the present invention as discussed herein. In particular, the computer programs, when executed, enable the processor102to perform the features of the computer system. Accordingly, such computer programs represent controllers of the computer system.