# STDP and STDP variations with memristors for spiking neuromorphic learning systems

T. Serrano-Gotarredona, T. Masquelier, T. Prodromakis, G. Indiveri, B.

Linares-Barranco

► To cite this version:

T. Serrano-Gotarredona, T. Masquelier, T. Prodromakis, G. Indiveri, B. Linares-Barranco. STDP and STDP variations with memristors for spiking neuromorphic learning systems. Frontiers in Neuroscience, 2013, 7, pp.2. 10.3389/fnins.2013.00002. hal-01578521

HAL Id: hal-01578521

https://hal.sorbonne-universite.fr/hal-01578521

Submitted on 29 Aug 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# STDP and STDP variations with memristors for spiking neuromorphic learning systems

T. Serrano-Gotarredona<fnref n="1" />, T. Masquelier<fnref n="2" />,<fnref n="3" />, T. Prodromakis<fnref n="4" />, G. Indiveri<fnref n="5" />
and B. Linares-Barranco<fnref n="1" />*

In this paper we review several ways of realizing asynchronous Spike-Timing-Dependent-Plasticity (STDP) using memristors as synapses. Our focus is on how to use individual memristors to implement synaptic weight multiplications, in a way such that it is not necessary to (a) introduce global synchronization and (b) to separate memristor learning phases from memristor performing phases. In the approaches described, neurons fire spikes asynchronously when they wish and memristive synapses perform computation and learn at their own pace, as it happens in biological neural systems. We distinguish between two different memristor physics, depending on whether they respond to the original "moving wall" or to the "filament creation and annihilation" models. Independent of the memristor physics, we discuss two different types of STDP rules that can be implemented with memristors: either the pure timing-based rule that takes into account the arrival time of the spikes from the pre- and the post-synaptic neurons, or a hybrid rule that takes into account only the timing of pre-synaptic spikes and the membrane potential and other state variables of the post-synaptic neuron. We show how to implement these rules in cross-bar architectures that comprise massive arrays of memristors, and we discuss applications for artificial vision.

**Keywords:** memristor/cmos, artificial-learning-synapses, spike-timing-dependent-plasticity, spiking-neural-networks

## 1. INTRODUCTION

For many years, the field of neuromorphic engineering has struggled to develop practical neuro-computing devices that mimicked the principles and operations of biological brains, by directly exploiting the physics of electronic devices in mixed analog/digital VLSI (Indiveri and Horiuchi, 2011). However, there always was a clamor for a compact and distributed non-volatile memory, possibly tightly coupled to the signal processing components (neurons), so

that the biological synapses counterparts could be properly emulated. The recent advent of nanoscale memristive-like devices (Wuttig and Yamada, 2007; Strukov et al., 2008; Yang et al., 2008; Jo et al., 2010; Govoreanu et al., 2011; Lee et al., 2011; Chanthbouala et al., 2012; Kuzum et al., 2012; Prodromakis et al., 2012a) opens the possibility of large-scale bio-inspired neural network implementations with minimal size-requirements for those elements in the circuit that are most numerous and therefore most

**Edited by:**

Gert Cauwenberghs, University of California, San Diego, USA

**Reviewed by:**

Emre O. Neftci, Institute of Neuroinformatics, Switzerland
Siddharth Joshi, University of California, San Diego, USA

**\*Correspondence:**

<figure><img src="image_3.png" /></figure>

B. Linares-Barranco, BSc Physics 1986, MSc 1987, PhD 1990 from University of Seville, Spain, and 2nd PhD 1991 from Texas A&M University, USA. He is Full professor of Research at the Instituto de Microelectrónica de Sevilla (IMSE-CNM-CSIC). He has been involved with circuit design for telecommunication circuits, VLSI emulators of biological neurons, VLSI neural based pattern recognition systems, hearing aids, precision circuit design for instrumentation equipment, bio-inspired VLSI vision processing systems, AER, memristors and emerging nanodevices, and VLSI transistor mismatch parameters characterization. He is IEEE Fellow.
bernabe@imse-cnm.csic.es

<footnote n="1">¹ Department of Analog and Mixed-Signal Design, Instituto de Microelectrónica de Sevilla, IMSE-CNM-CSIC, Sevilla, Spain</footnote>

<footnote n="2">² Unit for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain</footnote>

<footnote n="3">³ Laboratory of Neurobiology of Adaptive Processes, UMR 7102, CNRS - University Pierre and Marie Curie, Paris, France</footnote>

<footnote n="4">⁴ Centre for Bio-inspired Technology, Institute of Biomedical Engineering, Imperial College London</footnote>

<footnote n="5">⁵ Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland</footnote>
space-intense: plastic synaptic connections. The strength of a synaptic link between two neighboring neurons depends on its history and more explicitly by the overall amount of neurotransmitters that has been propagated through it after a relevant neural spike. In similar fashion, the strength of a **memristor**, i.e., its memristance (or instantaneous resistance) is dictated by the amount of charge *q* that has flown through it or the accumulated voltage flux *φ*. Additionally, the intrinsic non-linear nature of practical solid-state memristors resembles the behavior of neural synapses.

On the other hand, and independently of the new nanoscale devices availability, the neuromorphic engineering field evolved naturally toward circuits and systems exploiting spiking signal encoding, as in biology. For example, a large collection of spike-driven vision sensors have been reported, such as sensors for luminance (Culurciello et al., 2003; Chen et al., 2011), temporal contrast (Barbaro et al., 2002; Mallik et al., 2005; Chan et al., 2007a; Lichtsteiner et al., 2008; Leñero-Bardallo et al., 2011; Posch et al., 2011; Serrano-Gotarredona et al., 2013), **motion** (Kramer, 1996; Sarpeshkar et al., 1996; Ozalevli and Higgins, 2005), and **spatial contrast** (Ruedi et al., 2003; Zaghloul and Boahen, 2004; Costas-Santos et al., 2007; Massari et al., 2008; Leñero-Bardallo et al., 2010). **Spike-driven principles** have also been used for auditory systems (Sarpeshkar et al., 2005; Wen and Boahen, 2006, 2009; Chan et al., 2007b), competition and Winner-Take-All networks (Indiveri, 2000; Chicca et al., 2007; Oster et al., 2008), learning (Mill et al., 2011), classification (Mitra et al., 2009), fall detection (Fu et al., 2008), and systems distributed over wireless sensor networks (Teixeira et al., 2005; Massari et al., 2008). Apart from real-time sensing, spike-driven processing systems can produce extremely fast responses. Examples of spike-driven processing modules (chips) are those that, emulating biological neocortical structures, perform spatio-temporal feature extraction such as fixed-kernel (Venier et al., 1997; Choi et al., 2005) or programmable kernel (Serrano-Gotarredona et al., 2006; Camuñas-Mesa et al., 2011, 2012) 2D convolutions, and generic massive neural processing (Vogelstein et al., 2007; Fieres et al., 2008; Khan et al., 2008; Serrano-Gotarredona et al., 2009; Zamarreno-Ramos, 2012).

Unavoidably, the learning capability is one key characteristic that is required for building cognitive artificial neural systems. Recently proposed artificial neural processing systems spend

great resources for this task: the multi-million European initiative FACETS/BrainScales (Fieres et al., 2008) is developing a 200.000 neuron wafer<fnref n="1" /> where most of the silicon area is used for implementing **Spike-Timing-Dependent-Plasticity (STDP)** learning mechanisms in the synapses. The UK initiative SpiNNaker (Khan et al., 2008) based on multi-processors ARM technology has to use hybrid packaging technology in order to encapsulate two separate Silicon chips into each chip package: one chip is being the genuine SpiNNaker chip with 18 ARM<fnref n="2" /> CPUs, and the second chip being a commercial 128MB DRAM chip for local synaptic storage. Both the learning mechanisms and the storage of learned parameters require substantial silicon real-estate in traditional silicon-based chip technology.

However, the advent of new nanoscale technologies has shed new expectations, giving hopes for the development of ultra-compact, fast and efficient learning and storage mechanisms that may result in affordable, low power, compact, large scale, artificial neural systems (Wuttig and Yamada, 2007; Strukov et al., 2008; Yang et al., 2008; Jo et al., 2010; Govoreanu et al., 2011; Lee et al., 2011; Chanthbouala et al., 2012; Kuzum et al., 2012; Prodromakis et al., 2012a). A very promising new class of nanoscale devices is the one that comprises the so called *memristors* (Chua, 1971; Chua and Kang, 1976; Strukov et al., 2008; Borghetti et al., 2009; Jo et al., 2009, 2010), whose distinct characteristic is that they have *memory* while they operate like variable two-terminal *resistors*. It was recently postulated that such tiny nanoscale devices, when driven by appropriately shaped voltage pulses, could be embedded within traditional CMOS<fnref n="3" /> microchips, resulting in truly asynchronous<fnref n="4" /> artificial learning neural “tissue” equipped

**Memristor**

Two terminal electronic device which operates similar to a resistor, but whose resistance changes dynamically as the device is being used.

**Spike-Timing-Dependent-Plasticity (STDP)**

One type of learning rule for artificial synapses in spiking neural networks, where the synaptic update depends on the timing characteristics of individual spikes at the synapse terminals.

<footnote n="1">¹Wafer: microchips are fabricated on silicon wafers (with diameter ranging from 1" to about 18"), which can hold several hundreds or thousands of individual chips that are later on cut and encapsulated into chip packages. The FACETS/BrainScaleS project is a wafer-scale design, meaning that the wafer is not cut into individual chips, but the whole wafer is used as a unit circuit.</footnote>

<footnote n="2">²ARM stands for “Acorn Risk Microprocessor” and is the name of a company providing embedded microprocessors for a variety of more complex chips, like cellular phones, usb-sticks, etc.</footnote>

<footnote n="3">³CMOS stands for “Complementary Metal Oxide Semiconductor” and refers to the most standard technology used for microchip fabrication.</footnote>

<footnote n="4">⁴In asynchronous systems no clock is required, as opposed to conventional digital computing systems. Consequently, there is no centralized time keeper that enforces actions to happen in lock-step with each other.</footnote>
with STDP (Linares-Barranco and Serrano-Gotarredona, 2009b,a; Zamarreño-Ramos et al., 2011).

Although this still needs to be proven experimentally and all practical limitations are yet to be identified, while memristors are continuously being improved and optimized over many labs worldwide, the potential of building very dense hybrid memristive-CMOS learning systems is there. The resulting implementations can be extremely compact STDP-equipped systems, which contrast with pure CMOS-based attempts that either have resulted in physical STDP synapses consuming significant chip real-estate (Fieres et al., 2008) or complex computational work arounds in more algorithmic solutions (Rast et al., 2010; Davies et al., 2012). In this paper we quickly review the basic principles behind exploiting memristance for asynchronous STDP and extend the original findings to other types of STDP. In the next section we quickly review the memristor concept as well as some of the postulated physical mechanisms responsible for its operation. After this, section 3 summarizes STDP and some variation of it, as well as additive, multiplicative and quadratic STDP. Sections 4 and 5 review how to combine memristors with specific CMOS neurons to result in different types of STDP. Section 6 mentions an application in the context of artificial vision, and section 7 provides conclusions.

## 2. MEMRISTORS

Memristance was postulated in 1971 by Chua (1971) as the fourth missing canonical circuit element through his famous symmetry argument, illustrated in **Figure 1**. According to circuit theoretical fundamentals, there are four basic electrical quantities (Chua et al., 1987): (1) voltage difference between two terminals "v," (2) current flowing through into a device terminal "i," (3) charge flowing through a device terminal or integral of current $q = \int i(\tau)d\tau$, and (4) flux or integral of voltage $\phi = \int v(\tau)d\tau$. A two-terminal device is said to be canonical (Chua et al., 1987) if either two of the four basic electrical quantities are related by a static<fnref n="5" /> relationship, as shown in **Figure 1**. A resistor has a static relationship between terminal voltage $v$ and device current $i$, as shown in **Figure 1B**. A capacitor shows a static relationship between charge $q$ and voltage $v$, as shown in **Figure 1C**. An inductor has a static relationship between its current $i$ and flux $\phi$, as shown in **Figure 1D**. These three devices have been very well known since the origins of Electronics and Electricity. However, there are other possibilities for combining the

<figure><img src="image_4.png" /></figure>

<footnote n="5">⁵By "static" we mean it is not altered by changes of the above electrical quantities, or by their history, integrals, derivatives, etc. These "static" curves can, however, be time-varying if the change is caused by an external agent. For example, a motor driven potentiometer would have a "static" $i/v$ curve that is time varying.</footnote>
four basic electrical quantities: (q, i), (v, φ), and (q, φ). Ignoring the combinations of a quantity with its own time derivative leaves us with one single additional possibility: (q, φ). This reasoning led Chua to postulate the existence of a fourth basic two-terminal element, which he called the *Memristor*. *Memristors* behave as resistances in which the resistance changes through some of the basic electrical quantities, and is somehow *memorized*. The memristor would show a static relationship between charge *q* and flux *φ*, as shown in **Figure 1E**. If the *q* vs. *φ* relationship is linear, the *memristor* degenerates into a linear resistor. Although none of the so-far reported memristors can be described by a static constitutive relationship in the (q, φ) plane (and thus, strictly speaking, the 1971 fourth canonical element is still missing), they all fall within Chua's 1976 generalization of *Memristive Systems* (Chua and Kang, 1976). From here on we will use the term *memristor* for Chua's 1976 definition of *memristive system*. Consequently, the simple concept of *memristance* as defined in **Figure 1D** can be extended to refer to any device exhibiting resistive behavior (its *i/v* curves cross the origin) whose resistance can change through some of the four basic electrical quantities (or a combination of them, or their time derivatives or integrals, etc.), while at the same time exhibiting *memory* for that resistance. In that case, more elaborate mathematical descriptions are required (Chua and Kang, 1976).

Memristance has recently been demonstrated (with extraordinary impact among the research community) in nanoscale two-terminal devices, such as certain titanium-dioxide (Strukov et al., 2008; Borghetti et al., 2009; Prodromakis et al., 2011, 2012a) and amorphous Silicon (Jo et al., 2009) cross-point switches. However, memristive devices were reported earlier by other groups (Argall, 1968; Prodromakis et al., 2012b). Memristance arises naturally in nanoscale devices because small voltages can yield enormous electric fields that produce the motion of charged atomic or molecular species, changing structural properties of a device (such as its doping profile) while it operates. Its functional characteristic has been a pinched hysteresis loop in the i-v domain (Figures 2C,D); a signature that has been observed in various dissipative devices (Prodromakis et al., 2012b). Particularly nowadays various emerging resistive random-access memory (ReRAM) nano-devices (Chua, 2011), with one scaling extreme being the atomic switch (Terabe et al., 2005), are classified as

being memristors, and show attributes that resemble biological synapses (Ohno, 2011) providing exciting prospects for demonstrating neuromorphic applications (Avizienis et al., 2012). Hysteresis is typically noticed in systems-devices that possess certain inertia, causing the value of a physical property to lag behind changes in the mechanism causing it; manifesting memory (Pershin and Di Ventra, 2011). Particularly in the case of nanoscale memristors, this inertia has been ascribed to Joule heating (Fursina et al., 2009), the electrochemical migration of oxygen ions (Nian et al., 2007) and vacancies (Yang et al., 2008), the lowering of Schottky barrier heights by trapped charge carriers at interfacial states (Hur et al., 2010), the phase-change (Wuttig and Yamada, 2007), the formation/rupture of conductive filaments (Kwon et al., 2010), Yang et al. (2012) in a device's core, or even to some extent a combination of the aforementioned switching mechanisms.

Clearly, the impact of memristors is foreseen to be realized through their nanometric dimensions (see **Figure 2B** which is a cross section of one of the structures in **Figure 2A**), their capacity to store multiple bits or a continuum of information per element (**Figure 2E**) and the minuscule energy required to write distinct states, resulting in high spatial- and high storage-density well beyond the current state-of-the-art (Govoreanu et al., 2011). Nonetheless, the fact that the functional properties of such elements are associated with rate-limiting (frequency-dependent) electro- or thermo-dynamic changes that are contingent on both the present as well as the past environment, presents us with opportunities in exploiting them as novel computation elements.

By definition, memristors can be either voltage/flux driven or current/charge driven. Depending on the polarity of the set and reset potentials required to change resistive states (RS), the devices can be classified as unipolar (URS) or bipolar (BRS) (Schindler et al., 2007) and consequently, their circuit symbol must indicate somehow their polarity, as depicted in **Figure 3A**. Voltage/flux driven memristors can be described by (Chua and Kang, 1976)

$$i_{\text{MR}} = G(w, v_{\text{MR}}, t)v_{\text{MR}} \quad (1)$$

$$\dot{w} = f(w, v_{\text{MR}}, t) \quad (2)$$

while current/charge driven memristors would be described as (Chua and Kang, 1976)

$$v_{\text{MR}} = R(w, i_{\text{MR}}, t)i_{\text{MR}} \quad (3)$$

$$\dot{w} = f(w, i_{\text{MR}}, t) \quad (4)$$
<figure><img src="image_5.png" /></figure>

**FIGURE 2** | Solid-state TiO₂-based memristors fabricated at Imperial College London. (A) Microphotograph of a memristor cross-bar array, with a close-up SEM illustration of a single cell appearing in the inset of (A). (B) CHEMI-STEM map of a lamella cross-section of one of the devices shown in (A): blue denotes Pt (top and bottom electrodes) while green and red correspond to Ti and O₂ species (Prodromakis et al., 2012a). (C) Simulated and measured pinched hysteresis I-V characteristics

(absolute memristor current |I| vs. signed memristor voltage V)
(Prodromakis et al., 2011) in log scale or (D) linear scale, and (E) multi-state programming of a TiO₂-based memristor: read pulses are positive and small amplitude (1V) that do not alter the resistance (memristance) of the memristor, while successive set pulses have negative high amplitude (3V) and do progressively alter the resistance (memristance) of the memristor.

Here *w* represents some *structural* property parameter of the memristor. For example, in the 2008 HP paper (Strukov et al., 2008) the operation of the reported memristor was pos- tulated as described by the *moving wall model* depicted in **Figure 3B**. In this simplified model a

memristor of height *L*, sandwiched between two electrodes, has a low resistance region of height *w* and a high resistance region of height *L* − *w*. The memristor is considered to be divided into two regions. Both regions are separated by a boundary wall at position *w*, which moves up
<figure><img src="image_6.png" /></figure>

and down with the amount of charge that has flown through the memristor (in the case of being current/charge driven) or the accumulated flux (in case of being voltage/flux driven). The memristor would behave as two variable resistors in series. The total effective resistance of the memristor would be described by

$$R = R_{\text{ON}} \frac{w}{L} + R_{\text{OFF}} \left(1 - \frac{w}{L}\right) \quad (5)$$

This moving wall model can approximate phenomena like migration of oxygen ions (Nian et al., 2007) and vacancies (Yang et al., 2008), the lowering of Schottky barrier heights by trapped

charge carriers at interfacial states (Hur et al., 2010), and the phase-change in some PCM (phase change materials) devices (Wuttig and Yamada, 2007).

However, resistive switching effects in dielectric-based devices have normally been assumed to be caused by conducting filament formation across the electrodes, although the understanding and modeling of these phenomena remains controversial. As a matter of fact, some researchers are observing the formation and annihilation of nanoscale width conducting filaments in memristors (Kwon et al., 2010; Yang et al., 2012). Precise modeling of this phenomenon is still under
research (Shihong et al., 2012). However, let us here propose the following very simplified view to approximate this physical mechanism. **Figure 3C** illustrates schematically a memristor with several conducting filaments between the two electrodes. The number of filaments or their cross-sectional area would increase or decrease with memristor operation. Let us call now *w* the total cross sectional area of the effective conducting filaments at a given instant in time, and *S* the total cross section area of the memristor. The filaments present high conductivity (low resistivity), while the bulk presents much lower conductivity (high resistivity). All formed parallel filaments behave as one effective resistance of low resistance, while the rest of the bulk behaves as another higher resistivity resistor. Therefore, now the memristor behaves as two variable resistors in parallel. Consequently, its total conductance (inverse of resistance) could be described as

$$G = G_{\text{ON}} \frac{w}{S} + G_{\text{OFF}} \left(1 - \frac{w}{S}\right) \quad (6)$$

where $G_{\text{ON}}$ is the conductance per effective cross section area of the filaments, and $G_{\text{OFF}}$ is the conductance per effective cross section area of the filament-less bulk material. Parameter *w* would change from 0 to $w_{\text{max}}$, the maximum possible effective cross section area of total conducting filaments ($w_{\text{max}} \le S$).

This changing cross section description not only approximates filament formation/annihilation phenomena, but also some other gradual cross section area variations observed in some phase-change or ferroelectric-domains-based materials (Chanthbouala et al., 2012).

As we will highlight later in sections 4 and 5, whether a memristor is better described by the moving wall model or the filament formation/annihilation model, impacts severely on the resulting type of STDP learning mechanism. The latter yields an additive type of STDP, while the former results in a quadratic type STDP. Note that a memristor can be either voltage/flux or current/charge driven, independently of whether it is a “wall” or a “filament” memristor.

### 3. SPIKE-TIMING-DEPENDENT-PLASTICITY

STDP is the ability of natural or artificial synapses to change their strength according to the precise timing of individual pre- and/or post-synaptic spikes (Gerstner et al., 1993, 1996; Markram et al., 1997; Bi and Poo, 1998, 2001; Zhang et al., 1998; Feldman, 2000; Mu and

Poo, 2006; Cassenaer and Laurent, 2007; Jacob et al., 2007; Young, 2007; Finelli et al., 2008; Masquelier et al., 2008, 2009). A nice overview of STDP and its history can be found elsewhere (Sjöström and Gerstner, 2010). STDP learning in biology is inherently asynchronous and on-line, meaning that synaptic incremental update occurs while neurons and synapses transmit spikes and perform computations. This contrasts to more traditional learning rules, like backpropagation (Rojas, 1996), where first neurons and synapses perform signal aggregation and neural state update (we call this here “performing phase”) and then synaptic updates are computed and applied (we call this here “weight update phase”) alternating these two phases during training. Even early proposals for memristor-based STDP learning implementations used artificial time-multiplexing to alternate continuously and synchronously between “performing” and “weight update” phases (Snider, 2008), thus requiring global system-wide synchronization. This can become a severe handicap when scaling up systems to arbitrary size. Here we show a fully asynchronous implementation for memristor-based STDP where “performing” and “weight update” phases happen simultaneously in a natural manner, as in biology (Linares-Barranco and Serrano-Gotarredona, 2009b,a; Zamarreño-Ramos et al., 2011), where there is no need for any global synchronization. Other researchers have proposed variations around these ideas (Bichler et al., 2012a; Kuzum et al., 2012).

**Figure 3D** shows the change of synaptic strength (in percent) measured experimentally from biological synapses as function of relative timing $\Delta T = t_{\text{pos}} - t_{\text{pre}}$ between the arrival time $t_{\text{pre}}$ of a pre-synaptic spike and the time $t_{\text{pos}}$ of generation of a post-synaptic spike. Although the data shows stochasticity, we can infer an underlying interpolated function $\xi(\Delta T)$ as shown in **Figure 3E**.

$$\xi(\Delta T) = \begin{cases} a^{+}e^{-\Delta T/\tau^{+}} & \text{if } \Delta T > 0 \\ -a^{-}e^{\Delta T/\tau^{-}} & \text{if } \Delta T < 0 \end{cases} \quad (7)$$

For a causal pre to post spike timing relation ($\Delta T > 0$) the strength of the synapse is increased, while for an anti-causal relation ($\Delta T < 0$) it is decreased. In the case of synapses with negative synaptic strength (as in some artificial realizations), the reversed version shown in **Figure 3F** can be used. Microchip CMOS circuit implementations of STDP rules that follow the description of Equation (7) have been reported (Indiveri et al., 2006), which result in about 30

#### Artificial learning synapses

Artificially manufactured device that behaves similar to a biological synapse, e.g., it’s communication strength (or synaptic weight) changes as the device is used according to some learning rule.

#### Nanoscale artificial synapse

This is an artificial synapse made using some device whose dimensions are below the micron ($10^{-6}$ m).

#### Tunable STDP

STDP learning rule whose mathematical description can be made to change in time.
transistors per plastic synapse, thus demonstrat-
ing the very high cost of their hardware realiza-
tion. Let us call this *double-spike STDP*, since the
weight will be updated after the arrival of the
second spike (either pre- or post-synaptic).

Alternative variations of STDP have been
proposed that do not require the intervention of
both pre- and post-synaptic spikes (Brader et al.,
2007), resulting in slightly less complex circuit
implementations (Mitra et al., 2009). Let us call
this single-spike STDP, since the weight will be
updated after the arrival of pre-synaptic spikes
only. This single-spike STDP rule updates the
synaptic weight depending on the value of two
local neural soma state variables. The first one is
the membrane voltage V(t) and the second one
is an auxiliary state variable C(t) proportional
to the neuron's firing rate and equivalent to
the biological neuron's Calcium concentration,
which has the following dynamics

$$
\dot{C} = -\frac{C(t)}{\tau_C} + J_C \sum_i \delta(t - t_i) \quad (8)
$$

where $J_C$ represents the contribution of one
single post-synaptic spike and the time con-
stant $\tau_C$ is comparable to the STDP learning
window $\Delta T$. The synaptic weight variable $\xi$ is
updated only when a pre-synaptic spike occurs
at time $t_{\text{pre}}$. The synaptic strength is increased or
decreased by fixed size steps $|a^{\pm}|$ depending on
the instantaneous values of $V(t_{\text{pre}})$ and $C(t_{\text{pre}})$
with respect to a given set of global thresholds
$\{\theta_v, \theta_{\text{up}}^l, \theta_{\text{up}}^h, \theta_{\text{down}}^l, \theta_{\text{down}}^h\}$, as:

$$
\xi(t_{\text{pre}}) = \begin{cases} a^{+} & \text{if } V(t_{\text{pre}}) > \theta_v \text{ and} \\ \theta_{\text{up}}^{l} < C(t_{\text{pre}}) < \theta_{\text{up}}^{h} \\ -a^{-} & \text{if } V(t_{\text{pre}}) < \theta_v \text{ and} \\ \theta_{\text{down}}^{l} < C(t_{\text{pre}}) < \theta_{\text{down}}^{h} \end{cases} \quad (9)
$$

Additionally, in this model the synaptic strength drifts slowly toward its upper or lower bound depending on whether it is above or below an intermediate threshold.

Both types of STDP rules, double-spike and single-spike, are very expensive to implement in conventional CMOS microchips (Indiveri et al., 2006; Fieres et al., 2008; Khan et al., 2008; Mitra et al., 2009). However, as we will see in the next section, both can be implemented with just one memristor per synapse if appropriate peripheral signal conditioning neurons are used in **hybrid CMOS/memristor** realizations.

Independently on whether STDP is either
double-spike or single-spike, it is said to be
either additive, multiplicative or quadratic if it

additionally depends or not on the actual synap-
tic strength. If the STDP update is independent
of the actual synaptic strength, it is said to be
additive. Additive STDP requires the weight val-
ues to be bounded to an interval because weights
will stabilize at one of their boundary values
(van Rossum et al., 2000; Rubin et al., 2001).
If the synaptic update is proportional to actual
synaptic strength, it is called *multiplicative STDP*
and weights may stabilize to values interme-
diate to their boundaries (van Rossum et al.,
2000; Rubin et al., 2001; Gütig et al., 2003).
If the synaptic weight update is proportional
to the square of actual synaptic strength, we
call it *quadratic STDP* (Zamarreño-Ramos et al.,
2011).

4. MEMRISTORS AND CMOS NEURONS FOR
DOUBLE-SPIKE STDP

The more traditional double-spike STDP learn-
ing rule [as modeled by Equation (7)] can, in
theory, be implemented by (Zamarreño-Ramos
et al., 2011) (a) using a particular type of
voltage/flux driven memristor (Jo et al., 2010)
whose operation might be approximated by
Equation (2) with (see **Figure 3G**)

$$
f(v_{\text{MR}}) = \begin{cases} I_o \operatorname{sign}(v_{\text{MR}}) \left[ e^{|v_{\text{MR}}|/v_o} - e^{v_{\text{th}}/v_o} \right] & \text{if } |v_{\text{MR}}| > v_{\text{th}} \\ 0 & \text{otherwise} \end{cases} \tag{10}
$$

and bounded synaptic strength $w \in [w_{\min}, w_{\max}]$, while (b) providing appropriately shaped pre- and post-synaptic spikes available at both synapse (memristor) electrodes (Zamarreño-Ramos et al., 2011). For example, consider a pair of identical pre- and post-synaptic spikes with a shape resembling that of biological spikes, with an on-set duration $|t_{\text{ail}}^{+}|$ and a tail of duration $|t_{\text{ail}}^{-}|$, as shown in Figure 3H,

$$
\text{spk}(t) = \begin{cases} A_{\text{mp}}^{+} \frac{e^{t/\tau^{+}} - e^{-t_{\text{ail}}^{+}/\tau^{+}}}{1 - e^{-t_{\text{ail}}^{+}/\tau^{+}}} & \text{if } -t_{\text{ail}}^{+} < t < 0 \\ -A_{\text{mp}}^{-} \frac{e^{-t/\tau^{-}} - e^{-t_{\text{ail}}^{-}/\tau^{-}}}{1 - e^{-t_{\text{ail}}^{-}/\tau^{-}}} & \text{if } 0 < t < t_{\text{ail}}^{-} \\ 0 & \text{otherwise} \end{cases} \quad (11)
$$

Under these circumstances, memristor voltage is
$v_{\text{MR}}(t, \Delta t) = \alpha_{\text{pos}} \text{spk}(t) - \alpha_{\text{pre}} \text{spk}(t + \Delta t)$ and
from Equations (2, 10) synaptic strength update
can be computed as

$$
\Delta w(\Delta T) = \int f(v_{\text{MR}}(t, \Delta T)) dt = \xi(\Delta T) \qquad (12)
$$
which has been shown to result in the same shape illustrated in **Figure 3E** (Zamarreño-Ramos et al., 2011). Furthermore, by reshaping the spike waveform one can fine tune or completely alter the STDP learning function ξ(Δt), as illustrated in **Figure 4** (Linares-Barranco and Serrano-Gotarredona, 2009a). This way, by building neurons with a given degree of shape programmability, it is possible to change the STDP learning function at will, depending on the application, or make it evolve in time as learning progresses.

**Figure 5A** shows a way of interconnecting memristors and CMOS neurons for STDP learning. Triangles represent the neuron soma, being the flat side its input (dendrites) and the sharp side the output (axon). Dark rectangles are memristors, representing each one synaptic

junction. Each neuron controls the voltage at its input (Vpost in **Figure 5B**) and output (Vpre in **Figure 5B**) nodes. When the neuron is not spiking it forces a constant voltage at both nodes, while collecting through its input node the sum of input synaptic spike currents coming from the memristors, which contribute to changing the neuron internal state. When the neuron spikes, it sets a one-spike waveform at both input and output nodes. This way, they send their output spikes forward as pre-synaptic spikes for the destination synaptic memristors, but also backward to preceding synaptic memristors as post-synaptic spikes. Zamarreño et al. showed extensive simulations on these concepts, and how one can change from STDP to anti-STDP by switching polarities of spikes or memristors (Zamarreño-Ramos et al., 2011).

<figure><img src="image_7.png" /><figcaption>**FIGURE 4** | Illustration of influence of action potential shapes on the resulting STDP memristor weight update function ξ(ΔT). Memristor upper and lower thresholds are normalized to amplitudes ±1.0. From (A1,A2) to (E1,E2) the same spike waveform travels forward and backward. In (F1,F2) the forward and backward waveforms are the same but have opposite</figcaption></figure>

polarity. In (G1,G2) to (H1,H2) the forward and backward waveforms are different. In (G1,G2), the positive pulse of the backward waveform exceeds amplitude +1.0, thus producing negative STDP update whenever there is a post-synaptic spike alone (G2); otherwise if pre- and post-synaptic spikes happen within a given time window, there will be positive STDP update.
<figure><img src="image_8.png" /></figure>

For example, **Figures 4F1, F2** illustrate the case where forward and backward spikes have opposite polarities, resulting in a symmetric STDP update function $\xi(\Delta T)$. **Figures 4G1, G2** illustrate an example where forward and backward spikes are different, with the backward spike such that its positive part exceeds the positive memristor threshold ($v_{th} = 1.0$). This produces LTD (long term depression) or negative STDP update whenever there is a post-synaptic spike sufficiently apart from a pre-synaptic one; and produces LTP (long term potentiation) if pre- and post-synaptic spikes happen within a given time window (Bichler et al., 2012b,a). **Figures 4H1, H2** illustrate a similar STDP update behavior, except that update (whether positive or negative) is restricted to a constraint time window.

If the system is structured into neural layers (for example, **Figure 5A** shows a 3-neuron-layer system) with **memristive synapses** in between, then for each layer all pre-synaptic neurons should have the same forward spike shape and all post-synaptic neurons should have the same backward shape. This way, all memristive synapses between these two neural layers will have the same STDP function $\xi(\Delta T)$.

## 4.1. ADDITIVE OR QUADRATIC STDP WITH MEMRISTORS

In all these circuits, synaptic strength is the conductance $G$ of the memristor: the higher the conductance of a memristor $G$ is (or the lower its resistance $R = 1/G$ is) the stronger the synaptic efficiency will be, as it will let more current through and thus affect more strongly the destination neuron state. Therefore, if the memristors used obey a “moving wall” model (see Equation 5), then STDP update $\Delta w = \xi(\Delta T)$ changes wall position $w$, which from Equation (5) is directly proportional to resistance

$$ \begin{aligned} \Delta R(\Delta T) &= (R_{\text{ON}} - R_{\text{OFF}}) \frac{\Delta w(\Delta T)}{L} \\ &= \rho \xi(\Delta T) \end{aligned} \quad (13) $$

where $\rho$ is a constant. Consequently, synaptic strength $G = 1/R$ will change as

$$ \begin{aligned} \Delta G(\Delta T) &= -\frac{\Delta R(\Delta T)}{R^2} \\ &= -G^2 \Delta R(\Delta T) \propto -G^2 \rho \xi(\Delta T) \end{aligned} \quad (14) $$

This means that synaptic strength update would follow a quadratic STDP learning rule.
If the memristor physics is better represented by the inter-electrode filament formation/annihilation model, then synaptic update would change parameter w of Equation (6), which is now directly proportional to memristor conductance (synaptic strength),

$$ \Delta G(\Delta T) = \frac{G_{\text{ON}}}{S} \Delta w(\Delta T) = \gamma \xi(\Delta T) \quad (15) $$

where γ is a constant. Therefore, synaptic update would be independent of actual weight (conductance) and the resulting STDP update rule is said to be of additive type. Note that Equations (10–12) and the resulting functions ξ(ΔT) in Figure 4 are common for both “wall” and “filament” models.

## 5. MEMRISTORS AND CMOS NEURONS FOR SINGLE-SPIKE STDP

For the case of the alternative single-spike STDP rule [as defined by Equations (8, 9)] we can use the same circuit topology shown in Figures 5A,B, but with different neuron circuits and spike shapes. Figure 5C shows one memristor connected between a pre-synaptic neuron generating voltage $V_{\text{pre}}(t) = spk(t)$ and a post-synaptic neuron that sets a given DC level at $V_{\text{pos}}(t)$. Figures 5D,E show possible waveforms for the output spike spk(t). It must be such that its amplitude is confined below the thresholds $\pm v_{\text{th}}$ of Figure 3G. We distinguish two parts: two sequential opposite sign square pulses during a time $t_{\text{stdp}}$, and one synapse activation waveform for neural integration of duration $t_{\text{nint}}$, with $t_{\text{stdp}} \ll t_{\text{nint}}$. During times $t_{\text{stdp}}$ the synapses connected to $V_{\text{pre}}$ may experience weight update. During times $t_{\text{nint}}$ the post-synaptic neurons would add the contribution of this pre-synaptic spike to their internal integrated state. The CMOS neuron in Figure 5C can be designed containing a current sensing circuit (made of the opamp with resistive feedback) which sets the voltage at the neuron input node $V_{\text{pos}}$. This current sensing circuit collects all currents provided by all synaptic memristors connected to the neuron input node $V_{\text{pos}}$. The total instantaneous current sensed drives blocks $V(t)$ and $C(t)$ which compute, respectively, the neuron membrane voltage and the calcium variable (see Equation 8). These two instantaneous state variables are monitored by block “DC level” which generates a DC output level of three possible values: either zero, positive or negative, according to Equation (9). This DC level is copied to node $V_{\text{pos}}$ only during times $t_{\text{stdp}}$ whenever some pre-synaptic neuron

starts to spike. If $V_{\text{pos}} = 0$ then memristor voltage is equal to spk(t) during time $t_{\text{stdp}}$ of a pre-synaptic spike production and is confined within the thresholds in Figure 5D. Under these circumstances, no synaptic strength update is produced. However, if the DC level at $V_{\text{pos}}$ is set to either the positive or negative output value, the memristor voltage is either shifted up or down during $t_{\text{stdp}}$, and it will overpass one of the two thresholds, resulting in either an increment or decrement of synaptic strength update. The contribution of one spike to synaptic strength update is controlled by the height of the two first positive and negative steps of spk(t) during time $t_{\text{stdp}}$. Both have equal area and thus do not contribute to $V(t)$ or $C(t)$ if $V_{\text{pos}} = 0$. However, if $V_{\text{pos}} \neq 0$ the symmetry is broken and there would be an undesired contribution. For this reason, current integration at $V(t)$ and $C(t)$ needs to be inhibited during times $t_{\text{stdp}}$. During times $t_{\text{nint}}$ of spk(t), the waveform must be such that it always falls below the memristor thresholds and will not affect synaptic weight update. However, its area (shown in green in Figures 5D,E, and which can be made either positive or negative) will contribute to the change in $V(t)$ and $C(t)$. This way, the parameters that control synaptic update and shape of function $\xi(t_{\text{pre}})$ are fully decoupled from the parameters that control neural state variables update. This differs from the case of the conventional double-spike STDP, where these parameters are coupled (Zamarreño-Ramos et al., 2011).

In the case of single-spike STDP, as in the case of the double-spike STDP, if a “wall model” memristor is used the resulting STDP learning would be of quadratic type. Otherwise, using a “filament formation/annihilation” memristor results in additive STDP learning.

## 6. AN APPLICATION: EXTRACTING VISUAL FEATURES

Memristors can be used in unsupervised learning models of the visual cortex, and hence extract statistical structure from visual information without requiring supervised labeling. In a first attempt to simulate the early visual system, we used a simple feed-forward set up combining an artificial spiking retina (Lichtsteiner et al., 2008) and a spiking neural network mimicking V1 (Zamarreño-Ramos et al., 2011). The artificial retina sensed the external world in a continuous (frame-free) manner, and generated spikes that were asynchronously propagated, as they flowed in, through the feed-forward network. In the V1 layers, neurons were equipped with

**Spiking Neural Network**
Network of neurons that interchange information among them using “spikes.” Spikes are abstractions of biological spikes (also called action potentials). In some electronic spiking neural networks, spikes have similar waveform shapes than in biology, but normally in electronics systems spikes are much simpler being represented by a square digital pulse.
memristor-based quadratic STDP (simulated). As the system was exposed to natural stimuli, memristors gradually put strong weights on retinal ON- and OFF-center cells with receptive fields aligned in the visual space—because those had correlated spike times—leading to orientation selectivity, in accordance with Hubel and Wiesel's classic model (Hubel and Wiesel, 1959). It is worth mentioning that there was no absolute reference time such as a frame onset, yet information was encoded and decoded in the relative spike times. More recently, we have reproduced these results in a more biologically detailed model, which also included the lateral geniculate nucleus (Masquelier, 2012). Other researchers have followed similar paths with simpler STDP learning functions (as in Figure 4G2) (Bichler et al., 2012b) and proposed PCM-based hardware implementations (Bichler et al., 2012a). Future work will evaluate memristors in subsequent layers, mimicking higher order neurons. We expect that selectivity to more complex visual features will emerge.

Notably, biological hardware is incredibly slow: neurons cannot fire more than a few hundred spikes per second, and those impulses propagate on axons between neurons with a velocity of 1–2 m/s. Spike-driven chips and memristors could be several orders of magnitude faster, and thus could emulate the biological visual system much faster than real time. For example, switching times in the order of nano seconds have been demonstrated for some Hf-based resistive switches of 10 × 10 nm size (Govoreanu et al., 2011). This is particularly appealing for visual learning, which takes months, if not years, in humans. But there is no reason why memristors could not, for example, extract visual features from huge image databases in a few seconds… We thus speculate that this line of research will yield revolutionary results in the next decade.

## 7. SUMMARY AND DISCUSSION

In the present paper we reviewed ways of exploiting memristors to implement high density physical neural hardware equipped with STDP. We considered two types of memristor models, the “moving wall” model which

results in quadratic STDP, and the “filament formation/annihilation” model which results in a more conventional additive STDP. We also considered two types of STDP rules: the conventional double-spike rule and a more elaborate and biologically realistic single-spike rule. Finally, we briefly reviewed an application for artificial vision learning systems that mimics the operation of the visual cortex.

Large scale neural memristive STDP systems have not been built yet. As memristors are nano-scale devices, they will certainly suffer from significant inter-device parameter mismatch. Querlioz et al. (2011) have analyzed the impact of device parameter mismatch on the performance of STDP learning with memristors using a learning rule similar to the one shown in Figure 4G and have observed very smooth performance degradation even for parameter dispersions as high as 25–50%. Homeostasis, at the neural firing sensitivity level, can be a mechanism to help in compensating synaptic variability (Querlioz et al., 2011). Alternatively, certain STDP functions (like the one in Figure 4G) capable of firing with either one single pre- or post-synaptic spike can induce homeostasis as well (Sjöström and Gerstner, 2010).

In summary, synaptic behavior mismatch is an important concern, but researchers are already proposing possible solutions. It remains, however, to physically build such systems at a large scale and verify in situ their reliability, mismatch, and other non-ideal effects, and determine if the proposed solutions are sufficient to make them work reliably and efficiently.

## ACKNOWLEDGMENTS

This work was supported by Spanish grants from the Ministry of Economy and Competitividad TEC200-106039-C04-01/02 (VULCANO) (with support from the European Regional Development Fund) and PRI-PIMCHI-2011-0768 (PNEUMA) coordinated with the European CHIST-ERA program, and Andalusian grant TIC6091 (NANONEURO). T. Masquelier was supported by the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement 269459 (CORONET).

## REFERENCES

Argall, F. (1968). Switching phenomena in titanium oxide thin films. *Solid-State Electron.* 11, 535–541.

Avizienis, A. V., Sillin, H. O., Martin-Olmos, C., Shieh, H. H., Aono, M., Stieg, A. Z., et al. (2012).

Neuromorphic atomic switch networks. *PLoS ONE* 7:e427772. doi: 10.1371/journal.pone.0042772

Barbaro, M., Burgi, P. Y., Mortara, A., Nussbaum, P., and Heitger, F. (2002). A 100x100 pixel silicon retina for gradient extraction with steering filter capabilities and

temporal output coding. *IEEE J. Solid State Circ.* 37, 160–172.

Bi, G., and Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. *J. Neurosci.* 18, 10464–10472.

Bi, G., and Poo, M. M. (2001). Synaptic modification by correlated activity: Hebb's postulate revisited. *Ann. Rev. Neurosci.* 24, 139–166.

Bichler, O., Suri, M., Querlioz, D., Vuillaume, D., DeSalvo, B., and Gamrat, C. (2012a). Visual pattern extraction using energy-efficient
“2-PCM Synapse” neuromorphic architecture. *IEEE Trans. Electron Devices* 59, 2206–2214.

Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J. P., and Gamrat, C. (2012b). Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity. *Neural Netw.* 32, 339–348.

Borghetti, J., Li, Z., Straznicky, J., Li, X., Ohlberg, D. A. A., Wu, W., et al. (2009). A hybrid nanomembristor/transistor logic circuit capable of self-programming. *PNAS* 106, 1699–1703.

Brader, J. M., Senn, W., and Fusi, S. (2007). Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. *Neural Comput.* 19, 2881–2912.

Camuñas-Mesa, L., Acosta-Jiménez, A., Zamarreño-Ramos, C., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 32 × 32 convolution processor chip for address event vision sensors with 155ns event latency and 20Meps throughput. *IEEE Trans. Circ. Syst.* I 58, 777–790.

Camuñas-Mesa, L., Zamarreño-Ramos, C., Linares-Barranco, A., Acosta-Jimenez, A. J., Serrano-Gotarredona, T., and Linares-Barranco, B. (2012). An event-driven multi-kernel convolution processor module for event-driven vision sensors. *IEEE J. Solid State Circ.* 47, 504–517.

Cassenaer, S., and Laurent, G. (2007). Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. *Nature* 448, 709–713.

Chan, V., Jin, C., and van Schaik, A. (2007a). An address-event vision sensor for multiple transient object detection. *IEEE Trans. Biomed. Circ. Syst.* I, 278–288.

Chan, V., Liu, S.-C., and van Schaik, A. (2007b). AER EAR: a matched silicon cochlea pair with address event representation interface. *IEEE Trans. Circ. Syst.* I 54, 48–59.

Chanthbouala, A., Garcia, V., Cherifi, R. O., Bouzehouane, K., Fusil, S., Moya, X., et al. (2012). A ferroelectric memristor. *Nat. Mater.* 11, 860–864.

Chen, D. G., Matolin, D., Bermak, A., and Posch, C. (2011). Pulse-modulation imaging – review and performance analysis. *IEEE Trans. Biomed. Circ. Syst.* 5, 64–82.

Chicca, E., Whatley, A. M., Lichtsteiner, P., Dante, V., Delbrück, T., Del Giudice, P., et al. (2007). A multichip pulse-based neuromorphic

infrastructure and its application to a model of orientation selectivity. *IEEE Trans. Circ. Syst.* I 54, 981–993.

Choi, T. Y. W., Merolla, P., Arthur, J., Boahen, K., and Shi, B. E. (2005). Neuromorphic implementation of orientation hypercolumns. *IEEE Trans. Circ. Syst.* I 52, 1049–1060.

Chua, L. (2011). Resistance switching memories are memristors. *Appl. Phys. A* 102, 765–783.

Chua, L. O. (1971). Memristor – the missing circuit element. *IEEE Trans. Circ. Theory* 18, 507–519.

Chua, L. O., Desoer, C. A., and Kuh, E. S. (1987). *Linear and Nonlinear Circuits*. New York, NY: McGraw-Hill.

Chua, L. O., and Kang, S. M. (1976). Memristive devices and systems. *Proc. IEEE* 64, 209–223.

Costas-Santos, J., Serrano-Gotarredona, T., Serrano-Gotarredona, R., and Linares-Barranco, B. (2007). A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems. *IEEE Trans. Circ. Syst.* I 54, 1444–1458.

Culurciello, E., Etienne-Cummings, R., and Boahen, K. A. (2003). A biomorphic digital image sensor. *IEEE J. Solid State Circ.* 38, 281–294.

Davies, S., Galluppi, F., Rast, A., and Furber, S. (2012). A forecast-based STDP rule suitable for neuromorphic implementation. *Neural Netw.* 32, 3–14.

Feldman, D. (2000). Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. *Neuron* 27, 45–56.

Fieres, J., Schemmel, J., and Meier, K. (2008). “Realizing biological spiking network models in a configurable wafer-scale hardware system,” *IEEE International Joint Conference on Neural Networks (IJCNN-WCCI)* (Hong Kong), 969–976.

Finelli, L. A., Haney, S., Bazhenov, M., stopfer, M., and Sejnowski, T. J. (2008). Synaptic learning rules and sparse coding in a model sensory system. *PLoS Comput. Biol.* 4:e1000062. doi: 10.1371/journal.pcbi.1000062

Fu, Z., Delbrück, T., Lichtsteiner, P., and Culurciello, E. (2008). An address-event fall detector for assisted living applications. *IEEE Trans. Biomed. Circ. Syst.* 2, 88–96.

Fursina, A. A., Sofin, R. G. S., Shvets, I. V., and Natelson, D. (2009). Origin of hysteresis in resistive switching in magnetite is Joule heating. *Phys. Rev. B* 79:245131. doi: 10.1103/PhysRevB.79.245131

Gerstner, W., Kempter, R., Leo van Hemmen, J., and Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. *Nature* 383, 76–78.

Gerstner, W., Ritz, R., and Hemmen, J. L. (1993). Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. *Biol. Cybern.* 69, 503–515.

Govoreanu, B., Kar, G. S., Chen, Y., Paraschiv, V., Kubicek, S., Fantini, A., et al. (2011). “10 × 10 nm² Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation,” in *Electron Devices Meeting (IEDM), 2011 IEEE International (Washington, DC)*, 31.6.1–31.6.4.

Gütig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. *J. Neurosci.* 23, 3697–3714.

Hubel, D. H., and Wiesel, T. N. (1959). Receptive fields of single neurones in the cat's striate cortex. *J. Physiol.* 148, 574–591.

Hur, J. H., Lee, M.-J., Lee, C. B., Kim, Y.-B., and Kim, C.-J. (2010). Modeling for bipolar resistive memory switching in transition-metal oxides. *Phys. Rev. B* 82:155321. doi: 10.1103/PhysRevB.82.155321

Indiveri, G. (2000). Modeling selective attention using a neuromorphic analog VLSI device. *Neural Comput.* 12, 2857–2880.

Indiveri, G., Chicca, E., and Douglas, R. (2006). A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. *IEEE Trans. Neural Netw.* 17, 211–221.

Indiveri, G., and Horiuchi, T. K. (2011). Frontiers in neuromorphic engineering. *Front. Neurosci.* 5:118. doi: 10.3389/fnins.2011.00118

Jacob, V., Brasier, D. J., Erchova, I., Feldman, D., and Shulz, D. E. (2007). Spike-timing-dependent synaptic depression in the *in vivo* barrel cortex of the rat. *J. Neurosci.* 27, 1271–1284.

Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., and Lu, W. (2010). Nanoscale memristor device as synapse in neuromorphic systems. *Nano Lett.* 10, 1297–1301.

Jo, S. H., Kim, K.-H., and Lu, W. (2009). High-density crossbar arrays based on a Si memristive system. *NANO Lett.* 9, 870–874.

Khan, M., Lester, D., Plana, L., Rast, A., Jin, X., Painkras, E., et al. (2008). Spinnaker: mapping neural networks onto a massively-parallel chip

multiprocessor. *IEEE International Joint Conference on Neural Networks (IJCNN-WCCI)* (Hong Kong), 2849–2856.

Kramer, J. (1996). Compact integrated motion sensor with three-pixel interaction. *IEEE Trans. Pattern Anal. Mach. Intell.* 18, 455–460.

Kuzum, D., Jeyasingh, R. G. D., Lee, B., and Wong, H.-S. P. (2012). Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. *NANO Lett.* 12, 2179–2186.

Kwon, D.-H., Kim, K. M., Jang, J. H., Jeon, J. M., Lee, M. H., Kim, G. H., et al. (2010). Atomic structure of conducting nanofilaments in TiO₂ resistive switching memory. *Nat. Nanotech.* 5, 148–153.

Lee, M.-J., Lee, C. B., Lee, D., Lee, S. R., Chang, M., Hur, J. H., et al. (2011). A fast, high-endurance and scalable non-volatile memory device made from asymmetric Ta₂O₅ – x/TaO₂ – x bilayer structures. *Nat. Mater.* 10, 625–630.

Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2010). A five-decade dynamic range ambient-light-independent calibrated signed-spatial-contrast AER retina with 0.1ms latency and optional time-to-first-spike mode. *IEEE Trans. Circ. Syst.* I 57, 2632–2643.

Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 3.6 µs latency asynchronous frame-free event-based dynamic vision sensor. *IEEE J. Solid State Circ.* 46, 1443–1455.

Lichtsteiner, P., Posch, C., and Delbrück, T. (2008). A 128 × 128 120 dB 15 µs latency asynchronous temporal contrast vision sensor. *IEEE J. Solid State Circ.* 43, 566–576.

Linares-Barranco, B., and Serrano-Gotarredona, T. (2009a). “Exploiting memristance in adaptive asynchronous spiking neuromorphic nanotechnology systems,” in *9th IEEE Conference on Nanotechnology (IEEE-NANO 2009)* (Genoa), 601–604.

Linares-Barranco, B., and Serrano-Gotarredona, T. (2009b). Memristance can explain spike-time-dependent-plasticity in neural synapses. Available from Nature Precedings http://hdl.handle.net/10101/npre.2009.3010.1

Mallik, U., Clapp, M., Choi, E., Cauwenberghs, G., and
Etienne-Cummings, R. (2005). "Temporal change threshold detection imager," in *Solid-State Circuits Conference, 2005. Digest of Technical Papers. ISSCC 2005 IEEE International, Vol. 1* (San Francisco, CA), 362–603.

Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS. *Science* 275, 213–215.

Masquelier, T. (2012). Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model. *J. Comput. Neurosci.* 32, 425–441.

Masquelier, T., Guyonneau, R., and Thorpe, S. J. (2008). Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. *PLoS ONE* 3:e1377. doi: 10.1371/journal.pone.0001377

Masquelier, T., Guyonneau, R., and Thorpe, S. J. (2009). Competitive STDP-based spike pattern learning. *Neural Comput.* 21, 1259–1276.

Massari, N., Gottardi, M., and Jawed, S. (2008). "A 100 µW 64 × 128-pixel contrast-based asynchronous binary vision sensor for wireless sensor networks," in *Solid-State Circuits Conference, 2008. ISSCC 2008. Digest of Technical Papers. IEEE International* (San Francisco, CA), 588–638.

Mill, R., Sheik, S., Indiveri, G., and Denham, S. L. (2011). A model of stimulus-specific adaptation in neuromorphic analog VLSI. *IEEE Trans. Biomed. Circ. Syst.* 5, 413–419.

Mitra, S., Fusi, S., and Indiveri, G. (2009). Real-time classification of complex patterns using spike-based learning in neuromorphic VLSI. *IEEE Trans. Biomed. Circ. Syst.* 3, 32–42.

Mu, Y. and Poo, M. M. (2006). Spike timing-dependent LTP/LTD mediates visual experience-dependent plasticity in a developing retinotectal system. *Neuron* 50, 115–125.

Nian, Y., Strozier, J., Wu, N., Chen, X., and Ignatiev, A. (2007). Evidence for an oxygen diffusion model for the electric pulse induced resistance change effect in transition-MetalOxides. *Phys. Rev. Lett.* 98, 146–403.

Ohno, T. (2011). Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. *Nat. Mater.* 10, 591–595.

Oster, M., Wang, Y., Douglas, R., and Liu, S.-C. (2008). Quantification of a spike-based winner-take-all VLSI network. *IEEE Trans. Circ. Syst.* I 55, 3160–3169.

Ozalevli, E., and Higgins, C. M. (2005). Reconfigurable biologically inspired visual motion system using modular neuromorphic VLSI chips. *IEEE Trans. Circ. Syst.* I 52, 79–92.

Pershin, Y. V., and Di Ventra, M. (2011). Memory effects in complex materials and nanoscale systems. *Adv. Phys.* 60, 145–227.

Posch, C., Matolin, D., and Wohlgenannt, R. (2011). A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. *IEEE J. Solid State Circ.* 46, 259–275.

Prodromakis, T., Peh, B. P., Papavassiliou, C., and Toumazou, C. (2011). A versatile memristor model with nonlinear dopant kinetics. *IEEE Trans. Electron Devices* 58, 3099–3105.

Prodromakis, T., Salaoru, I., Khiat, A., and Toumazou, C. (2012a). "Concurrent resistive and capacitive switching of nanoscale TiO₂ memristors," in *Nature Conference on Frontiers in Electronic Materials: Correlation Effects and Memristive Phenomena* (Aachen, Germany).

Prodromakis, T., Toumazou, C., and Chua, L. O. (2012b). Two centuries of memristors. *Nat. Mater.* 11, 478–481.

Querliaz, D., Bichler, O., and Gamrat, C. (2011). "Simulation of a memristor-based spiking neural network immune to device variations," in *Proceedings of the International Joint Conference on Neural Networks (IJCNN)* (San Jose, CA), 1775–1781.

Rast, A., Galluppi, F., Davies, S., and Furber, S. (2010). "Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware," in *The 2010 International Joint Conferences on Neural Networks (IJCNN)* (Barcelona).

Rojas, R. (1996). *Neural Networks: A Systematic Introduction*. Berlin: Springer.

Rubin, J., Lee, D. D., and Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric heuristic plasticity. *Phys. Rev. Lett.* 86, 364–367.

Ruedi, P.-F., Heim, P., Kaess, F., Grenet, E., Heitger, F., Burgi, P.-Y., et al. (2003). A 128 × 128 pixel 120-dB dynamic-range vision sensor chip for image

contrast and orientation extraction. *IEEE J. Solid State Circ.* 38, 2325–2333.

Sarpeshkar, R., Baker, M. W., Salthouse, C. D., Sit, J.-J., Turicchia, L., and Zhak, S. M. (2005). "An analog bionic ear processor with zero-crossing detection," in *Solid-State Circuits Conference, 2005. Digest of Technical Papers. ISSCC. 2005 IEEE International, Vol. 1* (San Francisco, CA), 78–79.

Sarpeshkar, R., Kramer, J., Indiveri, G., and Koch, C. (1996). Analog VLSI architectures for motion processing: from fundamental limits to system applications. *Proc. IEEE* 84, 969–987.

Schindler, C., Thermadam, S. C. P., Waser, R., and Kozicki, M. N. (2007). Bipolar and unipolar resistive switching in cu-doped SiO₂. *IEEE Trans. Electron Devices* 54, 2762–2768.

Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jimenez, A., and Linares-Barranco, B. (2006). A neuromorphic cortical-layer microchip for spike-based event processing vision systems. *IEEE Trans. Circ. Syst.* I 53, 2548–2566.

Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gomez-Rodriguez, F., et al. (2009). CAVIAR: a 45 k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. *IEEE Trans. Neural Netw.* 20, 1417–1438.

Serrano-Gotarredona, T., and Linares-Barranco, B. (2013). A 128 × 128 1.5% contrast sensitivity 0.9% FPN 3us latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance amplifiers. *IEEE J. Solid-State Circuits* (in press).

Shihong, M. W., Prodromakis, T., Salaoru, I., and Toumazou, C. (2012). Modelling of current percolation channelsemergingresistiveswitching elements. arXiv:1206.2746v1 [cond-mat.mes-hall].

Sjöström, J., and Gerstner, W. (2010). Spike-timing dependent plasticity. *Scholarpedia* 5:1362. doi: 10.4249/scholarpedia.1362

Snider, G. S. (2008). "Spike-timing-dependent learning in memristive nanodevices," in *IEEE International Symposium on Nano Architectures* (Anaheim, CA), 85–92.

Strukov, D. B., Snider, G. S., Stewart, D. R., and Williams, R. S. (2008). The missing memristor found. *Nature* 453, 80–83.

Teixeira, T., Andreou, A. G., and Culurciello, E. (2005). "Event-based imaging with active illumination in sensor networks," in *Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS)* (Kobe), 644–647.

Terabe, K., Hasegawa, T., Nakayama, T., and Aono, M. (2005). Quantized conductance atomic switch. *Nature* 433, 47–50.

van Rossum, M. C. W., Bi, G. Q., and Turrigiano, G. G. (2000). Stable hebbian learning from spike timing-dependent plasticity. *J. Neurosci.* 20, 8812–8821.

Venier, P., Mortara, A., Arreguit, X., and Vittoz, E. A. (1997). An integrated cortical layer for orientation enhancement. *IEEE J. Solid State Circ.* 32, 177–186.

Vogelstein, R., Mallik, U., Vogelstein, J., and Cauwenberghs, G. (2007). Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. *IEEE Trans. Neural Netw.* 18, 253–265.

Wen, B., and Boahen, K. (2006). "A 360-channel speech preprocessor that emulates the cochlear amplifier," in *IEEE ISSCC Digest of Technical Papers* (San Francisco, CA), 556–557.

Wen, B., and Boahen, K. (2009). A silicon cochlea with active coupling. *IEEE Trans. Biomed. Circ. Syst.* 3, 444–455.

Wuttig, M., and Yamada, N. (2007). Phase-change materials for rewritable data storage. *Nat. Mater.* 6, 824–832.

Yang, J. J., Pickett, M. D., Li, X., Ohlberg, D. A. A., Stewart, D. R., and Williams, R. S. (2008). Memristive switching mechanism for metal/oxide/metal nanodevices. *Nat. Nanotech.* 3, 429–433.

Yang, Y., Gao, P., Gaba, S., Chang, T., Pan, X., and Lu, W. (2012). Observation of conducting filament growth in nanoscale resistive memories. *Nat. Commun.* 3:732. doi: 10.1038/ncomms1737

Young, J. M. (2007). Cortical reorganization consistent with spike timing-but not correlation-dependent plasticity. *Nat. Neurosci.* 10, 887–895.

Zaghloul, K. A., and Boahen, K. (2004). Optic nerve signals in a neuromorphic chip I & II. *IEEE Trans. Biom. Eng.* 51, 657–675.

Zamarreno-Ramos, C., Camuñas-Mesa, L. A., Pérez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011).
On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. *Front. Neurosci.* 5:26. doi: 10.3389/fnins.2011.00026

Zamarreno-Ramos, C., Linares-Barranco, A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2012). Multicasting mesh AER: a scalable assembly approach for reconfigurable neuromorphic structured AER systems. application to ConvNets. *IEEE Trans. Biomed.*

*Circ. Syst.* Available online at: http://ieeexplore.ieee.org/xpl/article Details.jsp?arnumber=6211459

Zhang, L., Tao, H., Holt, C., Harris, W., and Poo, M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. *Nature* 395, 37–44.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any

commercial or financial relationships that could be construed as a potential conflict of interest.

Received: 13 October 2012; accepted: 06 January 2013; published online: 18 February 2013.

Citation: Serrano-Gotarredona T, Masquelier T, Prodromakis T, Indiveri G and Linares-Barranco B (2013) STDP and STDP variations with memristors for spiking neuromorphic learning systems. *Front.*

Neurosci. 7:2. doi: 10.3389/fnins. 2013.00002

Copyright © 2013 Serrano-Gotarredona, Masquelier, Prodromakis, Indiveri and Linares-Barranco. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.