<figure><img src="image_1.png" /></figure>

# Excitatory and inhibitory STDP jointly tune feedforward neural circuits to selectively propagate correlated spiking activity

Florence I. Kleberg*, Tomoki Fukai and Matthieu Gilson

Lab for Neural Circuit Theory, RIKEN Brain Science Institute, Wako, Japan

**Edited by:**
Cristina Savin, IST Austria, Austria

**Reviewed by:**
Maoz Shamir, Boston University, USA
Paul Miller, Brandeis University, USA
Ashok Litwin-Kumar, University of Pittsburgh, USA

**\*Correspondence:**
Florence I. Kleberg, Lab for Neural Circuit Theory, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako, Saitama, 351-0198, Japan
e-mail: kleberg@brain.riken.jp

Spike-timing-dependent plasticity (STDP) has been well established between excitatory neurons and several computational functions have been proposed in various neural systems. Despite some recent efforts, however, there is a significant lack of functional understanding of inhibitory STDP (iSTDP) and its interplay with excitatory STDP (eSTDP). Here, we demonstrate by analytical and numerical methods that iSTDP contributes crucially to the balance of excitatory and inhibitory weights for the selection of a specific signaling pathway among other pathways in a feedforward circuit. This pathway selection is based on the high sensitivity of STDP to correlations in spike times, which complements a recent proposal for the role of iSTDP in firing-rate based selection. Our model predicts that asymmetric anti-Hebbian iSTDP exceeds asymmetric Hebbian iSTDP for supporting pathway-specific balance, which we show is useful for propagating transient neuronal responses. Furthermore, we demonstrate how STDPs at excitatory-excitatory, excitatory-inhibitory, and inhibitory-excitatory synapses cooperate to improve the pathway selection. We propose that iSTDP is crucial for shaping the network structure that achieves efficient processing of synchronous spikes.

**Keywords:** STDP, spike-timing, plasticity, inhibition, disynaptic, correlation, excitation-inhibition balance

## 1. INTRODUCTION

Activity-dependent plasticity of synaptic connections between neurons is crucial for cortical circuit development and memory (Böhme et al., 1993; Hensch et al., 1998). Spike-timing-dependent plasticity (STDP) describes the change in synaptic weights where long-term potentiation (LTP) and long-term depression (LTD) depend on the precise timing of presynaptic and postsynaptic action potentials. STDP has been observed for excitatory glutamate-mergic synapses in a great diversity of brain structures, such as the hippocampus (Magee and Johnston, 1997; Bi and Poo, 1998; Debanne et al., 1998), the cerebellum of the electric fish (Bell et al., 1997), the neocortex (Markram et al., 1997; Sjöström et al., 2001), and the optic nerve in Xenopus (Zhang et al., 1998). An extensive body of theoretical work has uncovered many interesting properties of excitatory STDP (eSTDP): it can select input pathways based on their spike-time correlation (Kempter et al., 1999; Song et al., 2000; Gjorgjieva et al., 2011), it can generate a stable distribution of weights (van Rossum et al., 2000; Gütig et al., 2003; Gilson and Fukai, 2011), it can perform selection of phase-locking in population firing (Gerstner et al., 1996; Senn and Buchs, 2003), it favors the emergence of functional neuronal assemblies (Izhikevich et al., 2004; Clopath et al., 2010), it stabilizes slow oscillations in recurrent networks (Kang et al., 2008) and it allows for rewiring of connections in the developing visual cortex (Song and Abbott, 2001; Senn and Buchs, 2003; Young et al., 2007).

There is also evidence for STDP at inhibitory GABAergic synapses, or iSTDP, (Woodin et al., 2003; Haas et al., 2006;

Kodangattil et al., 2013). However, our understanding of the mechanistic implications of iSTDP remains limited, in spite of the key role of inhibition in signal processing in the cortex (van Vreeswijk and Sompolinsky, 1996; Anderson et al., 2000; Wehr and Zador, 2003; Haider et al., 2006, 2013; Maffei et al., 2006; Rudolph et al., 2007). Considering the abundance of inhibitory interneurons, e.g., in the cortex (Markram et al., 2005), remarkably few types have been tested for the plastic properties of their synapses. Theoretical knowledge of the dynamics and functional implications of iSTDP is also rudimentary, although interest in this direction has increased recently. Extending a simple homeostatic control of firing rate, iSTDP can generate a balance between excitatory and inhibitory inputs onto a neuron (Vogels et al., 2011). In addition, unspecific, but sufficiently strong inhibition developed by iSTDP can enhance competition between excitatory synapses subject to eSTDP (Luz and Shamir, 2012). Interestingly, neither experimental nor theoretical approaches provide a consensus for the shape of the iSTDP learning window, in contrast to eSTDP for which the temporally Hebbian nature (LTP for pre-post pairing, LTD for post-pre pairing) is observed and addressed in the vast majority of cases.

The present paper aims to compare the effect of distinct iSTDP window shapes on the structure of synaptic weights, and endeavors to clarify the role of iSTDP in tuning neuronal responses. Previous studies have shown the precise timing of spikes to convey an important part of information about stimuli in sensory pathways (Riehle et al., 1997; Jackson et al., 2003;
Maldonado et al., 2008; Kilavik et al., 2009; Putrino et al., 2010). Moreover, neurons are sensitive to precise timings of spikes (Rossant et al., 2011). In this context of neural temporal coding, we examined the transmission of temporally correlated spikes in a feedforward circuit equipped with eSTDP and iSTDP. Such neural architectures with joint feedforward excitation and inhibition have been found in various brain structures (Buzsaki, 1984; Davis et al., 1996). We incorporated in our model an important property of the afferent inputs observed experimentally in many feedforward neural architectures: inhibition is delayed compared to excitation with a short time lag (Pouille and Scanziani, 2001; Wilent and Contreras, 2004; Gabernet et al., 2005; Silberberg and Markram, 2007; Tan et al., 2008; Stokes and Isaacson, 2010), which allows for precise temporal gating (Kremkow et al., 2010). We found that iSTDP with specifically anti-Hebbian properties enforces a balanced structure in the synaptic weights, which supports efficient processing of near-coincident spikes.

## 2. RESULTS

We examined the joint development of excitatory and inhibitory synapses subject to STDP in a feedforward circuit. We consider two circuit architectures. First, for a single neuron with direct excitatory and inhibitory inputs, we examine how the shape of the iSTDP window affects the evolution of synaptic weights. Since there is no current consensus about a single type of iSTDP (Woodin et al., 2003; Haas et al., 2006; Kodangattil et al., 2013), this comparison allows us to link the shape of iSTDP learning windows to their functional implications. Second, we examine the recruitment of interneurons in a more realistic circuit with monosynaptic excitation and disynaptic inhibition. In both cases, we focus on how the emerging weight structure tunes the propagation of spike volleys in the circuit.

### 2.1. THEORETICAL PREDICTION OF WEIGHT SPECIALIZATION DEPENDING ON iSTDP WINDOW

In order to study the weight dynamics for different iSTDP learning windows, we consider a simplified feedforward circuit (SFC) that consists of a single postsynaptic Poisson neuron excited by excitatory and inhibitory spike trains (Figure 1A). Following experimental observations, excitatory and inhibitory inputs have correlated spiking activity (Okun and Lampl, 2008). In addition, inhibition arrives with a delay *d* (Okun and Lampl, 2008; Atallah and Scanziani, 2009). The inhibitory delay mimics a disynaptic pathway, as compared to monosynaptic excitation (Figure 1B). The temporal correlations between spikes trains in Figure 1B are governed by the time constant $\tau_{in}$ (Figure 1C). All Synapses are plastic.

Excitatory weights are modified by a temporally Hebbian eSTDP rule (Gilson and Fukai, 2011), corresponding to the blue learning window in Figure 1D: a presynaptic spike preceding a postsynaptic spike leads to potentiation. The eSTDP update includes log-STD P weight dependence, which produces a long-tailed distribution of weights (Gilson and Fukai, 2011). For every pair of a pre- and a postsynaptic spikes, the weight $w_e$ is modified by a quantity that depends

on the current value of $w_e$ and the spike-time difference $\Delta t = t_{pre} - t_{post}$:

$$ \Delta w_e = W_e(\Delta t, w_e) = \begin{cases} \eta_e \exp\left(\frac{\Delta t}{\tau_{LTP}^e}\right) a_{LTP} \exp\left(-\frac{C_{LTP} w_e}{w_0}\right) & \text{for } \Delta t < 0, \\ \eta_e \exp\left(-\frac{\Delta t}{\tau_{LTD}^e}\right) a_{LTD} \exp\left(\frac{\log(1 + w_e/w_0 C_{LTD})}{\log(1 + C_{LTD})}\right) & \text{for } \Delta t > 0. \end{cases} \quad (1) $$

The time constants $\tau_{LTP}^e = 17$ ms and $\tau_{LTD}^e = 34$ ms and coefficients $a_{LTP} = 1$ and $a_{LTD} = -0.5$ determine the shape of the eSTDP window. $\eta_e$ is the learning rate. The log-style weight dependence scales the LTD curve and ensures a stable fixed point at $w_0 = 0.065$ for uncorrelated inputs; $C_{LTD} = 5$ enforces sufficiently strong competition between the incoming weights onto a given neuron. An exhaustive eSTDP parameter list is given in Table 1.

For inhibitory synapses, we test three types of additive iSTDP windows, shown in orange in Figure 1E:

* Hebbian (Haas et al., 2006; Luz and Shamir, 2012; Kodangattil et al., 2013), with pre-post LTP;

* Anti-Hebbian, with post-pre LTP;

* Symmetric (Vogels et al., 2011), with which LTP occurs for pre-post and post-pre spike pairings.

For every spike pair the inhibitory weight is updated with

$$ \Delta w_i = W_i(\Delta t) = \begin{cases} \eta_i p \exp\left(-\frac{\Delta t}{\tau_{post}^i}\right) & \text{for } \Delta t > 0, \\ \eta_i q \exp\left(\frac{\Delta t}{\tau_{pre}^i}\right) & \text{for } \Delta t < 0. \end{cases} \quad (2) $$

The right and left sides of the iSTDP window can be either LTP or LTD depending on the sign of *p* and *q*, respectively. Table 2 lists the values of *p* and *q* for the three windows employed in the theoretical model and in the simulations. For all iSTDP window types, total LTP exceeds total LTD; for anti-Hebbian, Hebbian, and symmetric (corrected), the difference LTP—LTD is set equal. Additionally, $\tau_{pre}^i = \tau_{post}^i$ for all iSTDP.

To stabilize iSTDP, each inhibitory weight is decreased by a small amount $\alpha$ for every presynaptic spike (Vogels et al., 2011), independently of the iSTDP contribution.

$$ W_i \rightarrow W_i - \eta_i \alpha \quad (3) $$

Our aim is to show the emergence of balance between excitation and inhibition through eSTDP and iSTDP. By balance we mean the simultaneous increase of excitatory and inhibitory weights, or weight balance, as opposed to increase of excitatory weights without increase of inhibitory weights. Balance known as the cancelation of currents onto a neuron (e.g., Vogels et al., 2011) can but need not follow from weight balance. Unless otherwise stated, balance in this study means weight balance. Using our analysis based on the Poisson neuron model (Materials and Methods), we evaluate the *expected change* in mean synaptic strengths for both sets of weights. The weight update is determined by the interplay of the iSTDP window, the input spike-time
A Simplified Feedforward Circuit (SFC) for theoretical analysis

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

**FIGURE 1** | Theoretical prediction of weight update for different types of iSTDP windows. (A) Schematic representation of the SFC. The Poisson neuron receives excitatory (light blue) and inhibitory (orange) inputs. All inputs are correlated with equal strength and a delay *d* for inhibition. (B) Example of the postsynaptic firing rate (black trace) in response to excitatory (light blue) and inhibitory (orange) spikes. (C) Left: Temporal cross-correlogram for a pair of similar input spike trains, excitatory or inhibitory. Lighter colors correspond to larger correlation spread τᵢₙ. Right, EPSP-IPSP shape including *d*. The asterisk indicates the convolution. (D) eSTDP window. The left (right) part in

the window indicates the presynaptic spike occurring before (after) the postsynaptic spike. (E) The three iSTDP learning windows: Hebbian, anti-Hebbian, symmetric. The expected weight change is the double convolution between the left and right figures in (C) and the window functions in (E). (F) Expected mean weight changes by analysis of a Poisson rate model for excitatory and inhibitory weights with simultaneous eSTDP and iSTDP. The plots correspond to the three iSTDP windows in (E). Warm colors represent inhibitory weights and cold colors excitatory weights. Lighter colors correspond to lighter colors in (C), namely larger τᵢₙ.

cross-correlograms and the postsynaptic response (EPSPs+IPSPs; **Figure 1C**). All expected weight changes in **Figure 1F** (and in subsequent sections, weights themselves) are shown after division by the excitatory equilibrium weight *w*₀, given in **Table 1**. The influence of the inhibitory delay *d* on the expected change in excitatory weights is weak—slight increase when *d* becomes larger—and does not depend much on the iSTDP learning window (**Figure 1F**, curves in cold colors). However, *d* affects

the evolution of inhibitory weights, as shown by the curves in hot colors in **Figure 1F**. For Hebbian iSTDP, inhibitory weights decrease with a stronger effect for larger delays (≥ 5 ms). Conversely, anti-Hebbian iSTDP causes weights to increase. Symmetric iSTDP leads to a potentiation that weakens for large delays.

In all cases, larger values for the input correlation width τᵢₙ decrease the effect of both eSTDP and iSTDP (curves in lighter
<table><caption>**Table 1 | General STDP parameters.**</caption><thead><tr><th>STDP parameters: SFC</th><th>Description</th><th>Value</th></tr></thead><tbody><tr><td>W<sub>e</sub>, W<sub>i</sub></td><td>eSTDP or iSTDP window</td><td></td></tr><tr><td>τ<sup>e</sup><sub>pre</sub>, τ<sup>e</sup><sub>post</sub></td><td>eSTDP window time constants</td><td>34 ms, 17 ms</td></tr><tr><td>τ<sup>i</sup><sub>pre</sub>, τ<sup>i</sup><sub>post</sub></td><td>iSTDP window time constants</td><td>30 ms</td></tr><tr><td>–</td><td>Excitatory start-up weights</td><td>random U[0, w<sub>0</sub> × 3]</td></tr><tr><td>–</td><td>Inhibitory start-up weights</td><td>0</td></tr><tr><td>w<sub>0</sub></td><td>Equilibrium weight for eSTDP</td><td>0.065</td></tr><tr><td>η<sub>e</sub></td><td>Excitatory learning rate</td><td>w<sub>0</sub> × 0.39</td></tr><tr><td>η<sub>i</sub></td><td>Inhibitory learning rate</td><td>0.075</td></tr><tr><td>α</td><td>inhibitory presynaptic single-spike contribution</td><td>0.2</td></tr><tr><td>C<sub>LTD</sub></td><td>LTD scaling for eSTDP</td><td>5</td></tr><tr><td>C<sub>LTP</sub></td><td>LTP scaling for eSTDP</td><td>50</td></tr><tr><td colspan="3"><strong>STDP PARAMETERS: FFC</strong></td></tr><tr><td>w<sub>0</sub></td><td>Equilibrium weight for eSTDP onto output neuron</td><td>0.08</td></tr><tr><td>η<sub>e</sub></td><td>Excitatory learning rate</td><td>w<sub>0</sub> × 0.78</td></tr><tr><td>η<sub>i</sub></td><td>Inhibitory learning rate</td><td>0.02</td></tr><tr><td>–</td><td>Excitatory start-up weights</td><td>random U[0, 1]</td></tr><tr><td>–</td><td>Excitatory-to-Inhibitory start-up weights</td><td>random U[0, 1]</td></tr><tr><td>–</td><td>Inhibitory start-up weights</td><td>1</td></tr><tr><td>τ<sup>i</sup><sub>pre</sub>, τ<sup>i</sup><sub>post</sub></td><td>iSTDP window time constants</td><td>20 ms</td></tr><tr><td>w<sup>i</sup><sub>0</sub></td><td>Equilibrium weight for eSTDP onto interneurons</td><td>w<sub>0</sub> × 2</td></tr><tr><td>–</td><td>Excitatory start-up weights onto interneurons</td><td>random U[0, 1]</td></tr></tbody></table>

Table listing all eSTDP and iSTDP parameters, with the exception of the iSTDP window parameters (Table 2). The notation "random U[a, b]" denotes a random distribution between a and b.

<table><caption>**Table 2 | iSTDP window parameters.**</caption><thead><tr><th>iSTDP type</th><th>Hebbian</th><th>anti-Hebbian</th><th>Symmetric</th><th>Symmetric (equal LTP/LTD)</th></tr></thead><tbody><tr><td>p</td><td>-1</td><td>1.5</td><td>1.5</td><td>0.25</td></tr><tr><td>q</td><td>1.5</td><td>-1</td><td>1.5</td><td>0.25</td></tr></tbody></table>

Table listing the different iSTDP windows. p and q indicate the amplitude of the right and left side of the iSTDP window, respectively.

colors in **Figure 1F**). In fact, Hebbian and anti-Hebbian iSTDP curves exhibit a delay for which the weight change is maximal. That “best” delay increases when $τ_{in}$ is large. Symmetric iSTDP is less affected by $τ_{in}$.

In summary, the simultaneous strengthening of correlated excitatory and inhibitory inputs (i.e., the emergence of balance) should occur when iSTDP has an anti-Hebbian LTP component in this simple circuit (anti-Hebbian and symmetric iSTDP), and

when inhibitory input spikes arrive after postsynaptic spikes with a sufficiently large d (axonal delay in a feedforward inhibitory circuit).

## 2.2. EMERGENCE OF A DETAILED BALANCE BETWEEN EXCITATORY AND INHIBITORY WEIGHTS

Next, we verify our theoretical predictions for the SFC with simulations using a LIF neuron (Materials and Methods: details of the simulated SFC, Equations 8, 9). In contrast to **Figure 1**, the SFC in **Figure 2A** includes a distractor pathway with random, uncorrelated inputs (**Figure 2A**, dark blue and red lines) besides the correlated inputs (light blue and orange lines).

A typical example of synaptic weight evolution with anti-Hebbian iSTDP is shown in **Figure 2B1**. The weights from random inputs remain weak (dark blue and red traces), whereas the weights from the correlated excitatory inputs (light blue traces) and inhibitory inputs (orange traces) are strengthened, indicating the development of within-pathway balance, or *detailed balance* (Vogels and Abbott, 2009). In detailed balance, the excitatory and inhibitory inputs to strong weights on a postsynaptic neuron have correlated spike times (or spike rates, Vogels and Abbott, 2009). In contrast, when excitation is balanced with inhibition from a different signal pathway, excitation and inhibition are not necessarily correlated, which we may call *global balance*. Both types of balance will be evaluated in the sections below. The histograms of the final weight distributions of this example show the development of weight structure for excitation and inhibition. Excitatory weights exhibit a long-tail distribution that follows from the log-type weight dependence used for eSTDP (Gilson and Fukai, 2011) (**Figure 2B2**: top). The distribution of inhibitory weights has a long tail as well (**Figure 2B2**: bottom), but looks more bimodal for smaller $τ_{in}$ due to increased competition between the weights (not shown).

As in our analytical approach, we compared the effect of different iSTDP windows on the inhibitory weights of an inhibitory-excitatory pathway with correlated spike-times. The comparison of the Hebbian, anti-Hebbian, and symmetric iSTDP windows agrees with the theoretical predictions of expected drift in weights in **Figure 1F**. Correlated inhibitory weights increase with both symmetric (**Figure 2D**, black curves) and anti-Hebbian iSTDP (magenta curves). Their final equilibrium value depends on d (see also **Figure 2C**): short delays are preferred only by symmetric iSTDP (**Figure 2D**, black curves). Anti-Hebbian iSTDP leads to a larger increase in inhibitory weights than symmetric iSTDP for larger delays d, and small $τ_{in}$. We also test an additional version of symmetric iSTDP: apart from the window with the same maximal amplitude as Hebbian and anti-Hebbian iSTDP (**Figure 1E**, bottom; **Figure 2D**, black curves), we apply a symmetric rule with the same amount of LTP-LTD area (equalized symmetric iSTDP; gray curves). Since this equalized window only leads to very small changes, we conclude that the amplitude of LTP is the crucial factor, not the overall LTP/LTD ratio. Lastly, inhibitory weights vanish to zero with Hebbian iSTDP (red curves). These findings confirm that the neuron first becomes driven by the correlated excitatory inputs through eSTDP; then, when excitatory inputs dictate postsynaptic firing times, correlated inhibitory inputs follow up through iSTDP. The increase
A Simplified Feedforward Circuit (SFC)
for simulations

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

**FIGURE 2** | Development of the synaptic structure induced by simultaneous eSTDP and iSTDP (numerical simulation). (A) Schematic representation of the Simplified Feedforward Circuit Model (SFC). One leaky integrate-and-fire (LIF) neuron receives four different input groups, of which half have temporal correlations with time constant $τ_{in}$ (light blue and orange) and half are random spike trains (dark blue and red). Spike-time correlations arise from common variation of the firing rate. Inhibition arrives after excitation with a delay *d*. (B1) Example of weight evolution over time of a simulation using eSTDP and anti-Hebbian iSTDP. Excitatory weights (top), inhibitory weights (bottom). The delay is *d* = 3 ms, and input temporal precision $τ_{in}$ = 2.12 ms. (B2) Histograms of the distribution of all weights at

the end of the anti-Hebbian iSTDP simulation shown in (B1). (C) Mean inhibitory (top) and excitatory (bottom) weights from the correlated input group after learning. Each pixel corresponds to the average of 10 simulations with inhibitory delay *d* (x-axis) and input temporal resolution $τ_{in}$ (y-axis). (D) Mean inhibitory weight of the correlated inputs after learning with different iSTDP windows: Anti-Hebbian (magenta), Hebbian (red), symmetric (black, gray) for three values of $τ_{in}$. Each plot corresponds to a line in (C). (E) Final excitatory and inhibitory weights of the correlated pathway for anti-Hebbian (left) and symmetric (right) iSTDP, shown for all $τ_{in}$ and *d* = 6 ms. One dot represents the mean excitatory and mean inhibitory weight at the end of one trial. Delays are pooled within the same color.

in inhibitory weight is determined by *d*, the timing of inhibitory spikes to the excitatory spikes (and therefore to the output spikes), together with the shape of the iSTDP window. Note that we set $α$ such that weights from background inputs remain weak.

The potentiation of excitatory and inhibitory weights with both anti-Hebbian and symmetric iSTDP exhibits a balance between correlated excitation and inhibition, as illustrated in **Figure 2E**. Stronger excitatory weights induced by eSTDP are
counterbalanced by stronger inhibition due to iSTDP. This phenomenon depends on the input correlation precision $\tau_{in}$ (smaller values in darker color), but not significantly on the delay *d*. The matching is not linear and depends on the learning window.

In summary, we find that simulations with the LIF neuron confirm the theoretical results with the Poisson neuron. Detailed balance in the weights from the correlated pathway can arise if the iSTDP window is anti-Hebbian or symmetric, but not if it is Hebbian.

## 2.3. SHARPENING THE NEURONAL RESPONSE IN THE SFC

While detailed balance between excitatory and inhibitory weights can arise through anti-Hebbian or symmetric iSTDP (Figure 2), anti-Hebbian iSTDP may increase inhibition to the point where it will dominate over excitation for the postsynaptic neuron, as shown in Figure 3A. This follows partly because we use additive iSTDP, which strongly potentiates inhibitory weights (with our choice of parameters). Whether inhibition dominates (Rudolph et al., 2007) or not, the detailed balance weight specialization underlies the tuning of the SFC function in propagating spike volleys.

As $\tau_{in}$ governs the temporal width of input spike volleys, we evaluate $\tau_{out}$ for the postsynaptic response (Figure 3B). To do so, we detect volleys whose coincident spikes exceed a threshold as “events.” We then build a peristimulus time histogram (PSTH) of the postsynaptic spikes with respect to the input events (Figure 3B: right). An example for the simulation in Figure 3B with anti-Hebbian iSTDP is shown in Figures 3C1,C2 for two values of *d*.

The PSTH obtained from the simulations with specific inhibition (leading to detailed balance; red curve) is compared to two control conditions:

* the inhibitory weights are swapped to obtain unspecific inhibition (leading to global balance; green curve)

* the inhibitory inputs are omitted (black curve).

The three conditions are characterized by different mean firing rates. For $\tau_{in} = 2.12$ in the SFC with anti-Hebbian iSTDP, $\tau_{out} = 2.21$ sp/s for specific inhibition, 2.23 sp/s for unspecific inhibition, and 38.94 sp/s for excitation only.

The difference in response width $\tau_{out}$ between specific inhibition and the unspecific inhibition control represents the particular contribution of detailed balance on output response sharpening. Likewise, removing the inhibitory inputs from the circuit and taking the difference with the unspecific inhibition control should reveal the response sharpening due to the general presence of inhibition (global balance). To evaluate the relative change of spiking probability induced by the input stimuli, we normalize the PSTH with respect to the mean postsynaptic firing rate (see Materials and Methods for details). This gives a signal/noise ratio (SNR) for the output spikes following an event in this detection task. As can be seen in Figure 3C (right), both the specific inhibition (red) and unspecific inhibition (green) enhance the SNR. The postsynaptic response is even sharper with the specific inhibition circuit for small delays (Figure 3C1,

detailed balance, red curve). This occurs when inhibition is timed with excitation (arrow). For larger *d* as in Figure 3C2, 8 ms, this sharpening vanishes, as inhibition cannot arrive sufficiently early right after excitation. In that case, the performance is closer to that with unspecific inhibition (global balance, green curves). This sharpening is efficient for all $\tau_{in} = 0.71-5.66$ ms, in the range of the delay *d*, as illustrated in Figure 3D. Note that very small delays *d* prevent a proper weight structure from developing with anti-Hebbian iSTDP, thus the sharpening of the response fails (Figure 3D: top: red curve). The principle can be explained by the presence of inhibition lowering the output firing rate, which increases the SNR of the neuron's response (Figure 3E: top). Additionally, precisely timed inhibition coming right after excitation further sharpens the response and improves the SNR (Figure 3E: bottom). Figure 3F summarizes the performance of the sharpening by the emerged detailed weight balance, as compared to the global balance with unstructured inhibition or in the absence of inhibition.

In our model, symmetric iSTDP performed similarly to anti-Hebbian iSTDP (Figure 3G). A small difference is that global inhibition contributed slightly more to the sharpening of the response than the inhibition in detailed balance. The performance is a bit better for anti-Hebbian iSTDP because the weights grow stronger. Lastly, because the weight structure does not develop with Hebbian iSTDP, no significant difference in $\tau_{out}$ is observed (Figure 3H: left). Actually, the weights from the random input group are not weakened to zero in the Hebbian case (Supplementary Figure 1), so the unspecific inhibition control condition, in which weights between the two pathways are swapped, leads to a slightly better performance (Figure 3H: right).

We conclude that detailed balance, as achieved by anti-Hebbian and symmetric iSTDP, in combination with Hebbian eSTDP, can lead to the temporal restriction of a postsynaptic response to correlated input spikes. Brief delays in inhibition prove most beneficial for this sharpening, though the exact optimal delay is dependent on the input correlation precision $\tau_{in}$.

## 2.4. RECRUITMENT OF DISYNAPTIC INHIBITORY PATHWAY WITH DELAY SELECTION

Finally, we explicitly model inhibitory interneurons in our circuit in order to examine how they are recruited in a more realistic architecture. In our FFC model in Figure 4A, two correlated input pathways (dark and light blue) compete against each other. The inhibitory inputs contain heterogeneous axonal delays. Here we focus on anti-Hebbian iSTDP, which proved efficient in developing feedforward inhibition in the previous sections. All excitatory synapses are subject to eSTDP as in Figure 2. In the example simulation in Figure 4B, the excitatory weights onto the output neuron (top) specialize to the dark blue group (“winning group”) at the expense of the light blue group (“losing group”). Note that in general, each group has 50% chance of winning because we use sufficiently competitive eSTDP (Gilson and Fukai, 2011). The inputs onto the interneurons specialize in a similar fashion, as shown for two different examples in Figure 4B (middle and bottom).
A

<figure><img src="image_15.png" /></figure>

Exc. Spikes
Inhib. Spikes

<figure><img src="image_16.png" /></figure>

Excitatory
conductance

<figure><img src="image_17.png" /></figure>

Membrane (mV)

<figure><img src="image_18.png" /></figure>

Inhibitory
conductance

Time (s)

C1

<figure><img src="image_19.png" /></figure>

Raw PSTH

<figure><img src="image_20.png" /></figure>

Normalized PSTH

<figure><img src="image_21.png" /></figure>

C2

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

Time from event (ms)

E

unspecific inhibition:
global balance

<figure><img src="image_24.png" /></figure>

specific inhibition:
detailed balance

<figure><img src="image_25.png" /></figure>

F

<figure><img src="image_26.png" /></figure>

Sharpening
due to:

specific inhibition

G

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

H

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

Delay (ms)

D

<figure><img src="image_33.png" /></figure>

delay = 3 ms

τ out (ms)

<figure><img src="image_34.png" /></figure>

delay = 8 ms

<figure><img src="image_35.png" /></figure>

τ in (ms)

unspecific inhibition

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

**FIGURE 3** | Sharpening of the postsynaptic response by timed inhibition. (A) Neuronal state after learning. One second activity after training for the simulation in **Figure 2B**: raster plot of the input excitatory/inhibitory spikes (blue/red), excitatory/inhibitory conductance (blue/magenta), and voltage (black). (B) Schematic indicating the construction of the PSTH. Events are detected using the correlated excitatory inputs (blue). Then, postsynaptic spikes that occur in a given window around the event are counted (gray). Response efficiency is evaluated by the temporal width of the PSTH $τ_{out}$. (C1) Effect of inhibition on the response of the postsynaptic neuron to correlated events for $τ_{in} = 2.12$ ms and $d = 3$ ms. Left: example of raw PSTH for postsynaptic spike count. Comparison of detailed balance (red) with the control of global balance (green) and no inhibition (black). The arrow indicates incoming specific inhibition. Right:

signal/noise ratio (SNR) obtained by normalizing the PSTHs. (C2) Same as in (C1) but with $d = 8$ ms. (D) Response sharpening for different values of $τ_{in}$ and $τ_{out}$. The gray unit line represents instances where the output width and the input width are equal. Top: $d = 3$ ms. Bottom: $d = 8$ ms. Legend corresponds to (C). (E) schematic indicating the effect of detailed balance and global balance on the response shape. (F) anti-Hebbian iSTDP learning window and the contribution of detailed and global balance to the sharpening of the response. The difference in $τ_{out}$ is shown for varying $d$ (x-axis) and the input $τ_{in}$ (y-axis). Left: difference in $τ_{out}$ between detailed balance and global balance. Warm colors indicate the response is sharper through detailed balance compared to global balance. Right: difference in $τ_{out}$ between global balance and no inhibition. (G) Same as in (D) but for symmetric iSTDP. (H) Same as in (D) but for Hebbian iSTDP, where no detailed balance emerged.
<figure><img src="image_39.png" /><figcaption>A Full Feedforward Circuit (FFC)</figcaption></figure>

B

<figure><img src="image_40.png" /></figure>

C

<figure><img src="image_41.png" /></figure>

E

Schematic of weight evolution

<figure><img src="image_42.png" /></figure>

F

<figure><img src="image_43.png" /></figure>

$\tau$ in =
2.12 ms

<figure><img src="image_44.png" /></figure>

$\tau$ in =
3.53 ms

Time from event (ms)

G

<figure><img src="image_45.png" /></figure>

$\tau$ in (ms)

D

<figure><img src="image_46.png" /><figcaption>**FIGURE 4** | Selection of delays in a disynaptic pathway by iSTDP. (A)</figcaption></figure>

Schematic representation of the full feedforward circuit model (FFC). The postsynaptic neuron receives excitatory input from two correlated groups that compete. Inhibition onto this neuron is provided via 50 fast-spiking interneurons (orange circles). Each interneuron receives inputs from both excitatory groups. The interneurons have axonal delays between 0 and 9 ms. All synapses are plastic, with learning windows shown by the insets. (B) The evolution of excitatory weights from the two input groups onto the output neuron (top) and onto two of the 50 interneurons (middle, bottom) in one example trial. Weights from group 1 (dark blue inputs) increase beyond those from group 2 (light blue inputs). Here, group 1 is the “winning group.”

Interneuron 1 (23) receives more input from the dark (light) blue group. (C)

Example of weight evolution onto interneurons and the subsequent change in inhibitory synaptic weights during the simulation (after 20, 300, 1000 s). Each

dot represents one of the 50 interneurons. The x-axis indicates the difference in total weights between the two input groups onto the interneuron. The right (left) part corresponds to interneurons specializing to the dark (light) blue input group. The y-axis indicates the weight of the inhibitory synapse onto the postsynaptic neuron. (D) Inhibitory weights after learning depend on the axonal delays of interneurons (x-axis) and specialization of their input weights, in the FFC with heterogeneous delays (top; each horizontal line represents an average over 10 simulations), and in the FFC with homogeneous delays (bottom; each square represents 10 simulations). (E) Schematic of the recruitment of interneurons and the consequence on the inhibitory weights, leading to detailed balance. (F) SNR of the response to correlated events in the FFC with heterogeneous delays. Top: for $\tau_{in} = 2.12$ ms. Bottom: for $\tau_{in} = 3.54$ ms. (G) Relationship between $\tau_{in}$ and $\tau_{out}$ for the FFC model: specific inhibition (red), unspecific inhibition (green), and without inhibitory interneurons (black).
After the specialization of excitatory synapses, inhibitory
synapses start to become potentiated. We find that the structure
in the inhibitory synapses develops only for interneurons that
specialize to the same group as the output neuron (Figure 4C:
right part of last panel; Figure 4D, top). This is a consequence
of the correlation between the spike trains fired by the interneu-
rons that specialize to the winning group, and the output neu-
ron spike train. Conversely, interneurons that specialize to the
losing group do not match their spike times to postsynaptic
spikes, and their weights remain weak (Figure 4C: left part of last
panel). Inhibitory and excitatory inputs onto the output neuron
become correlated, making detailed balance possible (Figure 4C,
last panel; Figures 4D,E). The use of homogeneous delays in the
FFC still achieves detailed balance (Figure 4D, bottom) though
the difference in inhibitory weight is smaller. This is because there
is no competition between winner-recruited neurons of differ-
ent delays (Figure 4D bottom, left), and loser-recruited neurons
can more easily adjust their firing times to postsynaptic firing if
they receive a small amount of input from the winning group
(Figure 4D bottom, right).

Importantly, LTP in the inhibitory weights depends on the axonal delay of their interneurons, in a similar manner as the SFC (Figure 4D: left; Figure 2C: top). For broader input spike volleys with larger τin, short delays are not selected by anti-Hebbian iSTDP. This ensures that inhibition will not cut off the output response before sufficiently many inputs are integrated. Similarly, late-arriving inhibition does not affect the sharpening of the response, therefore there is no need for its weight to be increased for this function. The adequately timed inhibition that follows excitation results in a sharper response to correlated events in the FFC (Figures 4F,G). The comparison with unspecific inhibi-tion (global balance) for which inhibitory weights are swapped with interneurons specialized to the losing group confirms that precise timing between excitation and inhibition is important for the response sharpening (red curves versus green curves in Figures 4F,G). As in the SFC, the response to τin in the range of 1–5 ms benefits most from the detailed balance (Figure 4G).

To test the robustness of the FFC against noise, we modified the
FFC by adding random uncorrelated inputs into the interneurons
and the output neuron, and decreased the number of inputs from
the correlated pathways (Noisy Full Feedforward Circuit, Noisy
FFC; Supplementary Figure 2A). Detailed balance emerged as in
the FFC, inhibitory synapses showed delay-dependent potentia-tion (Supplementary Figure 2B), and the response from the out-
put neuron was sharpened (Supplementary Figures 2C,D, red
curve). Detailed balance and the sharpening role of inhibition are
therefore robust against noise.

We conclude that in the more realistic FFC and Noisy FFC,
eSTDP determines the specialization of both the output neu-
rons and the interneurons, and anti-Hebbian iSTDP selects the
interneurons with intermediate delays, which leads to sharpening
of the response.

**3. DISCUSSION**

This study showed how eSTDP and iSTDP can jointly structure
synapses in feedforward neural circuits to control downstream fir-
ing. We found that the temporally anti-Hebbian (post-pre LTP)

component of iSTDP is crucial to achieve a balance between
excitatory and inhibitory weights given correlated inputs, and
assuming an inhibitory delay in the order of a few milliseconds.
Moreover, interneurons can be recruited by Hebbian eSTDP in a
self-organized fashion to develop inhibition through iSTDP onto
output neurons. By selecting adequate delays in this disynaptic
inhibition scheme, iSTDP sharpens the output firing response,
enhancing the propagation of spike volleys.

**3.1. INPUT TIMING AND TYPES OF iSTDP LEARNING WINDOW**

We investigated how the interplay between eSTDP and iSTDP
shapes the excitatory and inhibitory weight distributions. In our
model, correlations in inhibition follow correlations in excitation
by a delay of up to 10 ms (Figures 1, 2, and 4), which agrees with
experimental observations at the order of a few milliseconds in
the auditory (Wehr and Zador, 2003) and somatosensory cortices
(Gabernet et al., 2005). For such input signals, we found that both
anti-Hebbian and symmetric iSTDP windows generate a detailed
balance between excitatory and inhibitory weights (see SFC in
Figure 2). In contrast, Hebbian iSTDP leads to the weakening of
all synapses: Due to the inhibitory delay and the timescale of the
input correlations, a large portion of the inhibitory spikes fall into
the LTD part of the window.

There is, to our knowledge, currently no experimental evi-
dence of this kind of anti-Hebbian iSTDP. Some studies show
evidence of anti-Hebbian STDP in excitatory synapses in the elec-
tric fish (Han et al., 2000; Harvey-Girard et al., 2010), in the
dorsal cochlear nucleus (Tzounopoulos et al., 2004), and in cor-
ticostriatal synapses (Fino et al., 2009). Anti-Hebbian STDP has
also been the subject of theoretical studies (e.g., Roberts and Bell,
2000; Rumsey and Abbott, 2004, 2006; Carnell, 2009), but again
only in the context of excitatory synapses. Our study is the first
one to show a functional role for the anti-Hebbian LTP in iSTDP.
Anti-Hebbian LTP is also part of the symmetric iSTDP learning
rule that is the subject of a recent theoretical iSTDP study by
Vogels et al. (2011), showing the exact balancing of excitation
and inhibition. In our model, the output neuron is dominated
by strong inhibition after learning, meaning that the balance
between excitatory and inhibitory weights leads to a different fir-
ing regime than in their results (Vogels et al., 2011). This follows
because of our choice of inputs, which induces strong LTP via
iSTDP.

Vogels et al. (2011) showed that symmetric iSTDP can lead
inhibitory feedforward connections to detailed balance with fixed
excitation, by letting inhibition adapt to the firing rate of each
input pathway. We propose that iSTDP can ensure pathway-
specific balance between excitation and inhibition, even if firing
rates are constant and excitation is growing simultaneously with
eSTDP. Since symmetric iSTDP contains an anti-Hebbian ele-
ment (namely, post-pre LTP), detailed balance will follow as long
as there is a positive delay in the inhibitory input (e.g., Pouille and
Scanziani, 2001; Wehr and Zador, 2003). Our theoretical results
show that the expected increase in weights does not depend on
input firing rate. If, however, firing rates are unequal between
input groups, we still expect our current results to hold, as long as
spike pairing-based effects dominate those coming from the rate
differences.
Our findings are in contrast with Hebbian iSTDP, which has been found experimentally in the entorhinal cortex (Haas et al., 2006) and in the ventral tegmental area (Kodangattil et al., 2013). If sufficient inhibition from other sources is present, synapses corresponding to uncorrelated inputs may be potentiated by Hebbian iSTDP, leading to a “reversed detailed balance”; a scenario in which inhibitory inputs from all but one pathway make up for the excitatory input from the remaining pathway. Although Hebbian iSTDP does not directly support detailed balance in the weights in our model, Hebbian iSTDP may subserve alternative functions in neural circuit processes. Recent theoretical work has shown that Hebbian iSTDP leads to decorrelation of inhibition with respect to excitation, which results in global balance and increased sensitivity to excitatory correlations (Luz and Shamir, 2012). This follows because of the increased sensitivity to input fluctuations when the neuron acts as a coincidence detector, in contrast to the integrator regime (Hong et al., 2012). Another study showed that Hebbian iSTDP also decorrelates spike patterns through lateral connections (Savin et al., 2010). Though these studies indicate that Hebbian iSTDP plays a part in creating global balance, it does not lead to the detailed balance in our feed-forward circuit. Alternatively, detailed balance by Hebbian iSTDP may arise if inhibitory delays are negative, for instance when somatic inhibitory inputs precede the excitatory dendritic spike. Inhibitory weight increase will, however, be strongly bounded by the fact that an early inhibitory spike may prevent a postsynaptic spike otherwise caused by late excitation, preventing weight increase.

Another form of inhibitory plasticity, slightly different from iSTDP considered here, is voltage-dependent iLTP (Maffei et al., 2006), which leads to a potentiation in inhibitory synapses when a presynaptic spike precedes a postsynaptic depolarization either without spikes (Maffei et al., 2006), or accompanied by low-frequency spiking (Wang and Maffei, 2014). Modeling approaches have shown that when iLTP is complemented by a homeostatic form of LTD, it is capable of creating sparseness in activation that supports stimulus-pair specificity in recipient neurons (Bourjaily and Miller, 2011a,b). iLTP contains a competitive effect for inhibitory synapses, meaning that the weakest synapses will not manage to decrease post-synaptic firing, therefore missing out on LTP. If the postsynaptic spiking rate is low, as in our study, we expect the inhibitory weight evolutions with iLTP to behave similarly to Hebbian iSTDP without the LTD part. This would not lead to detailed balance, because of the brief delay in inhibition, but global balance might ensue when implemented in a large network.

In view of the large diversity of inhibitory interneurons (Markram et al., 2005), explaining the possible roles of iSTDP in different circuits and interneurons is an important open question that requires further work.

## 3.2. RECRUITMENT OF DISYNAPTIC INHIBITORY PATHWAY IN FEEDFORWARD NETWORK

In our Full Feedforward Circuit model (FFC model), the excitation-inhibition structure in synaptic weights arises from the recruitment of interneurons: specialization due to eSTDP, followed by the strengthening of inhibition onto output neurons induced by iSTDP. Hebbian eSTDP provides a sufficient degree

of temporal correlation between the selected excitatory and inhibitory pathways onto the output neuron. This correlation is essential for anti-Hebbian iSTDP to select weights from adequate interneurons, whose firing is correlated with the output neuron.

One could also imagine other combinations of eSTDP-iSTDP for the interneurons in the FFC model. For example, if the eSTDP onto the interneurons is anti-Hebbian, excitation and inhibition onto the output neuron become *anti*-correlated. We expect that Hebbian iSTDP for the inhibitory synapses from the interneurons would be an interesting choice in this case, to further reinforce the anticorrelation between excitation and inhibition onto the output neuron.

## 3.3. CONTROL OF CORRELATED FIRING ACTIVITY

In the feedforward circuit, iSTDP enables the neuron to select inhibition with an adequate delay (Figure 4D), which temporarily controls the propagation of the volley of correlated spikes without arriving too early to stop it entirely (Figure 3D: top; Figures 3F,G). Moreover, the selected suitable delays depend on the input temporal precision ($\tau_{in}$): for temporally broader spike volleys, larger delays are recruited (Figure 4D). In this sense, the output firing is sharpened only after sufficiently many inputs have been integrated, in agreement with experimental findings (Gabernet et al., 2005).

It is worth noting that delayed inhibition compared to excitation arises naturally because of the disynaptic pathway (axonal delays of interneurons). For the inputs, we considered sharp correlations at the scale of a few milliseconds, in line with the timescale of input correlations for which neurons in a balanced state are most sensitive, shown both experimentally *in vitro* and theoretically by Rossant et al. (2011). Propagation of spike volleys in networks also requires such fine temporal resolution (Diesmann et al., 1999). Our results suggest that interneurons can control the temporal spread of such spike volleys by adapting an inhibitory cutoff. This function of iSTDP is complementary to the homeostatic stabilization (Pouille et al., 2009) enforced by iSTDP to control the average firing of neurons, as was demonstrated recently (Vogels et al., 2011). In addition to restraining the firing rate, iSTDP can control the temporal output by creating a detailed balance in the synaptic weights, in which precisely timed inhibition limits the output spikes to a narrow temporal window. Thus, our finding is in accordance with previous studies that show that inhibition limits the time for summation and integration of EPSCs (Pouille and Scanziani, 2001; Gabernet et al., 2005). The presence of inhibition improves frequency tuning in excitatory neurons in auditory cortex (Wu et al., 2008). We showed that sharpening of the response only takes place if the inhibitory delay is sufficiently brief. Such short delays can limit the range of intensity tuning in auditory neurons by reducing the EPSP amplitude, controlling the response integration window (Wu et al., 2006). Quick but delayed inhibition after excitation therefore allows only inputs from high intensities to generate spikes in the downstream neuron. The millisecond-range sharpening of the response by inhibition, such as in our model, may therefore be useful for tuning control of a neuron.

For certain delays and $\tau_{in}$, the well-timed inhibition may hyperpolarize the neuron so strongly that the responses exhibit a rebound, after inhibition vanishes (Supplementary Figure 3).
This is because the strong hyperpolarization brings the membrane potential far from the excitatory reversal potential, temporarily boosting subsequent excitatory inputs. We only found rebound responses for anti-Hebbian-based iSTDP, for which inhibitory weights grew strongest. Mechanisms to regulate inhibitory strength within a medium range could prevent this phenomenon, such as an *ad hoc* upper bound on inhibitory weights or weight-dependent iSTDP.

Finally, we also showed that response sharpening was robust to noise in the circuit, even when the correlated inputs were decreased, meaning that our results can be extended to more realistic circuit contexts with larger input numbers. iSTDP and its resulting structure in weights may therefore be useful for the propagation of transient activities in larger circuits, such as a cortical column.

# 4. MATERIALS AND METHODS

Here we provide details about our analysis to predict the weight changes induced by simultaneously occurring eSTDP and iSTDP in the first section of Results. Then we describe the two neural circuit architectures used in this study, namely the SFC and FFC. Finally, we explain how the PSTHs of the postsynaptic and output neurons are calculated.

## 4.1. THEORETICAL ANALYSIS OF THE WEIGHT EVOLUTION IN THE SIMPLIFIED FEEDFORWARD CIRCUIT (SFC)

In our theoretical model, a postsynaptic Poisson neuron post receives both excitatory and inhibitory inputs (Figure 1A). All inputs share the same source of correlation, and inhibition is delayed by *d* compared to excitation.

The firing rate $\rho_{post}$ evolves over time according to the presynaptic inputs:

$$ \rho_{post}(t) = \sum_{k} w_{k}^{e}[\epsilon_{e} * S_{k}^{e}](t) - \sum_{m} w_{m}^{i}[\epsilon_{i} * S_{m}^{i}](t). \quad (4) $$

The *k*th excitatory input spike train $S_k^e$ is modeled as a time series of Dirac functions: $S_k^e(t) = \sum_s \delta(t - t_s^k)$; likewise, $S_m^i$ is the *m*th inhibitory spike train. Though $\rho_{post}$ may take on negative values in theory, we assume it is positive on average, and do not consider the case of no postsynaptic spiking. The EPSPs and IPSPs are summed together to obtain $\rho_{post}; *$ denotes the convolution of functions. For each EPSP at synapse *k*, the time course of the postsynaptic response for a single spike is described by the normalized kernel functions $\epsilon_e$ rescaled by the weight $w_k^e$. For IPSP at synapse *m*, the same holds with $\epsilon_i$ and $w_m^i$. In Figure 1, we use a simple exponential decay that is identical for all excitatory synapses with decay time $\tau_e = 3$ ms; likewise $\tau_i = 5$ ms for all inhibitory synapses.

In order to evaluate the expected weight change, we calculate the pre-post spike-time correlations for excitatory and inhibitory inputs. We consider the situation when pre-post correlations are dominated by the effect of input correlations. Actually, we use spike-time covariances defined as (Gilson et al., 2010)

$$ C_{k,l}^{e,e}(t, \Delta t) = \mathrm{Cov}[S_k^e, S_l^e](t, \Delta t) := \langle S_l^e(t) S_k^e(t + \Delta t) \rangle \\ - \langle S_l^e(t) \rangle \langle S_k^e(t + \Delta t) \rangle. \quad (5) $$

The angular brackets $\langle \cdot \cdot \cdot \rangle$ denote the ensemble average over the randomness from the stochastic process. Considering spike trains with constant average firing rates and fixed pair-wise correlations, we can omit the dependence on *t* in Equation (5). For the configuration described in Figure 1A, excitatory inputs are homogeneously correlated between them, as well as inhibitory inputs. However, the correlation between an excitatory and an inhibitory inputs involves the delay *d*. Denoting by $C_0(\Delta t)$ the homogeneous covariance corresponding to Figure 1C, we have

$$ \begin{aligned} C_{k,l}^{e,e}(\Delta t) &= C_0(\Delta t), \\ C_{m,n}^{i,i}(\Delta t) &= C_0(\Delta t), \\ C_{k,m}^{e,i}(\Delta t) &= C_0(\Delta t - d). \end{aligned} \quad (6) $$

All covariances are defined in a similar manner to Equation (5). For the *k*th excitatory input, the covariance $C_{k,post}^e$ is given by the input covariance on which the postsynaptic response (EPSPs-IPSPs) operates:

$$ \begin{aligned} C_{k,post}^{e}(\Delta t) &:= \mathrm{Cov}[S_k^e, S_{post}^e](t, \Delta t), S_{post}^e \propto \rho_{post} \\ &= \mathrm{Cov} \left[ S_k^e \left( \sum_l w_l^e \epsilon_e * S_l^e - \sum_n w_n^i \epsilon_i * S_n^i \right) \right] (\Delta t) \\ &= \sum_l w_l^e [C_{k,l}^{e,e} * \epsilon_e](\Delta t) - \sum_n w_n^i [C_{k,n}^{e,i} * \epsilon_i](\Delta t) \quad (7) \\ &= \sum_l w_l^e [C_0 * \epsilon_e](\Delta t) - \sum_n w_n^i [C_0 * \epsilon_i](\Delta t - d). \end{aligned} $$

The subsequent STDP weight update is given by the integral value of the learning window $W_e(u)$ with the pre-post covariance $C_{k,post}^e(-u)$, which yields:

$$ \begin{aligned} \Delta w_k^e &= [C_{k,post}^e * W_e](0) \\ &= \sum_l w_l^e [C_0 * \epsilon_e * W_e](0) - \sum_n w_n^i [C_0 * \epsilon_i * W_e](d) \end{aligned} \quad (8) $$

Similarly, the pre-post covariance and the expected change for the *m*th inhibitory weight is given by:

$$ \begin{aligned} C_{m,post}^{i}(\Delta t) &= \sum_{l} w_{l}^{e}[C_{0} * \epsilon_{e}](\Delta t + d) - \sum_{n} w_{n}^{i}[C_{0} * \epsilon_{i}](\Delta t), \\ \Delta w_{m}^{i} &= \sum_{l} w_{l}^{e}[C_{0} * \epsilon_{e} * W_{i}](-d) \\ &\quad - \sum_{n} w_{n}^{i}[C_{0} * \epsilon_{i} * W_{i}](0) \end{aligned} \quad (9) $$

These formulas are used to generate Figure 1F.

## 4.2. DETAILS OF THE SIMULATED SFC

In Figures 2, 3, the SFC consists of a single postsynaptic neuron that receives a total of 200 excitatory and 50 inhibitory inputs. Half of each set of inputs consist of weakly correlated spike trains, whereas the remainder consists of random Poisson spike trains (Figure 2A).
We use a function in Brian Simulator to generate correlated spike trains, which is based on the first method in Brette (2008). The principle of this function is that a doubly stochastic process (or Cox Process) with an average (spike) rate $r$, underlies a group of inhomogeneous Poisson processes which have rates that fluctuate around $r$. Final spike trains are derived from these inhomogeneous Poisson processes, and will appear to be homogeneous, correlated spike trains with stationary rate $r$. These correlated spike trains do not have Poisson statistics, because their autocovariance is modulated by their correlation. In order to have exponential cross-correlation functions (CCF) between these spike trains, the function employs the Ornstein–Uhlenbeck process. The time-constant of the exponential CCF is a parameter called $\tau_c$ in Brette (2008) and in Brian Simulator. We focus on the standard deviation of the latencies in input spike volleys (representing input stimuli), $\tau_{in}$, where $\tau_{in} = \tau_c\sqrt{2}$. We apply correlation strength $c = 0.1$ and CCF standard deviation $\tau_{in}$ in the range of 0.71–5.66 ms. Correlated inhibition is delayed by $d$ ms. All inputs have the same firing rate $r_{in} = 5$ sp/s.

The postsynaptic neuron is a conductance-based leaky integrate-and-fire (LIF) model. Its membrane potential $V$ obeys:

$$\tau_m \frac{dV}{dt} = E_{\text{leak}} - V + g_e(E_e - V) + g_i(E_i - V) \quad (10)$$

With synaptic conductances $g_e$ and $g_i$, that decay exponentially with conductance trace parameters $\tau_e$ and $\tau_i$:

$$\tau_e \frac{dg_e}{dt} = -g_e, \quad \tau_i \frac{dg_i}{dt} = -g_i \quad (11)$$

For every excitatory spike from synapse $k$, $g_e$ is increased by $w_k^e$, and for every inhibitory spike from synapse $m$, $g_i$ by $w_m^i$. Intrinsic time constants of the neuron are not considered. All simulations are run with BRIAN, a python-based neural simulator (Goodman and Brette, 2008). The simulations last 2500 s each. For plots with error bars and color maps, 10 trials are repeated for the same simulation protocol with each set of values for $d$ and $\tau_{in}$. All SFC variables and parameters are listed in Table 3.

### 4.3. DETAILS OF THE SIMULATED FFC

In Figure 4, the FFC model incorporates 50 inhibitory interneurons which receive the same excitatory inputs as the output neuron, and project inhibitory connections onto the latter. In contrast to the SFC, two groups of correlated inputs compete against each other (dark blue and light blue lines in Figure 4A). The postsynaptic neuron receives 100 inputs from each group, and each interneuron receives 10 excitatory synapses from each group. The inputs are chosen so that the first interneuron receives excitatory input from spike trains 1–10 from the dark blue group, the second interneuron receives input from spike trains 2–11 from the dark blue group, and so on. The same procedure is performed for inputs from the light blue group. The 50 interneurons only differ from the output neuron by a shorter membrane time constant $\tau_m^i = 5$ ms. The interneurons are not connected to one another and there is no external inhibition source. Each interneuron makes a single inhibitory synapse onto the output neuron,

<table><caption>Table 3 | SFC and FFC variables and parameters.</caption><thead><tr><th>Theoretical SFC variables</th><th>Description</th><th>Value</th></tr></thead><tbody><tr><td>ρ<sub>post</sub></td><td>Postsynaptic neuron firing rate</td><td></td></tr><tr><td>s<sup>e</sup><sub>k</sub>, s<sup>e</sup><sub>l</sub></td><td>Presynaptic excitatory spiketrain k or 1</td><td></td></tr><tr><td>S'<sup>e</sup><sub>m</sub>, S'<sup>e</sup><sub>n</sub></td><td>Presynaptic inhibitory spiketrain m or n</td><td></td></tr><tr><td>ε<sub>e</sub></td><td>Synaptic conductance decay function for excitation</td><td></td></tr><tr><td>ε<sub>i</sub></td><td>Synaptic conductance decay function for inhibition</td><td></td></tr><tr><td>w<sup>e</sup><sub>k</sub>, w<sup>e</sup><sub>l</sub></td><td>Synaptic weight of excitatory input k or 1</td><td></td></tr><tr><td>w<sup>i</sup><sub>m</sub>, w<sup>i</sup><sub>n</sub></td><td>Synaptic weight of inhibitory input m or n</td><td></td></tr><tr><td>u</td><td>Time between two spikes</td><td></td></tr><tr><td>τ<sub>e</sub></td><td>Excitatory conductance decay constant</td><td>3 ms</td></tr><tr><td>τ<sub>i</sub></td><td>Inhibitory conductance decay constant</td><td>5 ms</td></tr><tr><td>d</td><td>Inhibitory delay</td><td>2–20 ms</td></tr><tr><td>τ<sub>c</sub></td><td>Time constant for input spike correlogram</td><td>0.5–4.0 ms</td></tr><tr><td>τ<sub>in</sub></td><td>STD of the latency of the input spike correlogram</td><td>τ<sub>c</sub> × √2</td></tr><tr><td>c</td><td>Correlation</td><td>0.1</td></tr><tr><td colspan="3"><strong>NUMERICAL EXTRA VARS: SFC and FFC</strong></td></tr><tr><td>V</td><td>Postsynaptic neuron membrane voltage</td><td></td></tr><tr><td>g<sub>e</sub></td><td>Excitatory synaptic conductance</td><td></td></tr><tr><td>g<sub>i</sub></td><td>Inhibitory synaptic conductance</td><td></td></tr><tr><td colspan="3"><strong>NUMERICAL EXTRA PARAMS: SFC and FFC</strong></td></tr><tr><td>τ<sub>in</sub></td><td>Input firing rate</td><td>5 Hz</td></tr><tr><td>τ<sub>m</sub></td><td>Membrane time constant</td><td>20 ms</td></tr><tr><td>τ<sub>i</sub></td><td>Inhibitory conductance decay constant</td><td>20 ms</td></tr><tr><td>E<sub>leak</sub></td><td>Leak Potential</td><td>−70 mV</td></tr><tr><td>E<sub>e</sub></td><td>Excitatory reversal potential</td><td>0 mV</td></tr><tr><td>E<sub>i</sub></td><td>Inhibitory reversal potential</td><td>−80 mV</td></tr><tr><td>–</td><td>LIF spike threshold</td><td>−50 mV</td></tr></tbody></table>

Table listing all SFC and FFC variables and parameters relating to inputs and neuronal properties.

and axonal delays $d$ are heterogeneous, ranging from 0 to 9 ms (five interneurons for each $d$).

All synapses are plastic. The excitatory synapses onto both the output neuron and the inhibitory interneurons are subject to the same Hebbian eSTDP leaning window. The list of parameters that vary from the SFC is shown in Table 2. The iSTDP window time constants are lower ($\tau_{pre}^i = \tau_{post}^i = 20$ ms), excitatory and inhibitory learning are slowed down ($\eta_e = 0.0624$, $\eta_i = 0.02$), the eSTDP equilibrium value is higher ($w_0 = 0.08$), and the value of the start-up weights is changed (a random number between 0 and 1 for excitation, 1 for inhibition). Total simulation time is 1000 s.

To test the robustness against noise, we modify the FFC by adding 400 excitatory random inputs onto the interneurons and output neuron (Noisy FFC in Supplementary Figure 2A, green inputs) and decrease the size of the correlated groups to 50 (dark and light blue inputs). The number of interneurons is increased to 120. Other parameters are unchanged (see Table 3 for all FFC parameters).
## 4.4. ANALYSIS OF THE TEMPORAL ACUITY OF THE POSTSYNAPTIC RESPONSE

We evaluate how the iSTDP learning rule, via the resulting weight distributions, shapes the postsynaptic response to correlated input activity. To do so we run the SFC simulation with fixed weights for 300 s.

Volleys of input spikes ("events") are detected by binning the spike times of the 100 correlated inputs in the SFC in bins with width 0.5 ms, and counting the spikes in a sliding window of duration $τ_{in}$. When the spike count exceeds a threshold, the time of the event is set to the center of the sliding window. Events in neighboring windows are discarded. The window spike count threshold is determined for each $τ_{in}$ such that the average number of events per second is as close as possible to $r_{in}$ without exceeding it.

To evaluate the temporal acuity of the spikes fired in response to such events, we count the postsynaptic spikes in bins of 0.5 ms. This yields a peri-stimulus time histogram (PSTHs) around the time of the events, as shown in **Figure 3B**. We then evaluate the temporal acuity of the response of the output neuron to input stimuli by computing the sharpness of the PSTH.

Not only the latencies of spikes following the event, but also the excess of spikes compared to the baseline output firing rate contributes to the temporal acuity of the response. We obtain the average firing rate during the entire 300 s simulation, $F_0$. The number of spikes in each bin of the PSTH is then divided by $F_0$, yielding the “normalized” PSTH as a deviation from average activity. This deviation is also the signal to noise ratio (SNR: **Figures 3C1, C2**). The temporal acuity of the response input events is then evaluated through the standard deviation of the normalized PSTH, $τ_{out}$. $τ_{out}$ is computed over the time window 0, +10 ms inside the PSTH (0 is the time of the event).

To study how the emerged inhibitory weight structure affects $τ_{out}$, we compare the outcome of simulations to two controls:

* “specific inhibition”: the weights are as in the numerical simulation of the SFC;

* “unspecific inhibition control”: the inhibitory weights are swapped between the two input groups, leading to equivalent total inhibition, but abolishing the relation between the temporal structure of the spike trains and the strength of the weights that depended on them through iSTDP;

* “excitation only control”: the inhibitory inputs are removed completely, leaving only the excitatory inputs.

In the unspecific inhibition control for the SFC, we aim to destroy the detailed weight structure that emerges, but preserve strong feedforward inhibition. After swapping the inhibitory weights between the correlated and random inputs, the weight strengths are adjusted down to obtain a postsynaptic firing rate similar to the specific inhibition configuration. Weight corrections are not performed for trials with mean weight smaller than 1. Excitatory weights are unchanged in all three conditions. In the FFC, the same three scenarios are applied. For specific inhibition, all weights are as obtained from the simulation. To obtain

the unspecific inhibition condition, excitatory weights onto the interneurons are swapped between the winner and loser input pathways. The result of this manipulation is that an interneuron receiving strong inputs from the dark blue group and weak inputs from the light blue group, changes to receiving weak inputs from the dark blue group and strong inputs from the light blue group. This procedure leads to qualitatively the same control as in the SFC. Inhibitory weights are not adjusted further as in the SFC. For the excitation only control, the interneurons are omitted.

## AUTHOR CONTRIBUTIONS

In this study, Matthieu Gilson and Florence I. Kleberg designed the experiments, Florence I. Kleberg performed the experiments and analyzed the data, and Florence I. Kleberg, Matthieu Gilson, and Tomoki Fukai wrote the paper.

## FUNDING

The research was funded by RIKEN.

## ACKNOWLEDGMENTS

The authors would like to thank Hideaki Shimazaki for helpful discussions, and Tom Sharp, Anthony DeConstanzo and Pierre Yger for their help writing the article.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fncom.2014.00053/abstract

**Supplementary Figure 1 | Equilibrium weights of the random inhibitory inputs in the SFC.** Mean inhibitory weight for Hebbian iSTDP (red curve), anti-Hebbian iSTDP (magenta curve), symmetric iSTDP (black curve), and symmetric with equal total LTP (gray curve). Mean final weights are shown for three $τ_{in}$.

**Supplementary Figure 2 | Robustness of weight structure development and response sharpening in the presence of noise.** (A) Noisy full feedforward circuit model with eSTDP and iSTDP. There are 120 interneurons instead of 50. The correlated input groups have decreased to 50 inputs each, and an additional 400 random inputs project onto the output neuron. Each interneuron also receives 60 random inputs. For eSTDP, $w_0 = 0.037$. Other parameters are as in the FFC. (B) Delay-dependent inhibitory weight strengthening of interneurons recruited by the winning group (top) and absence of inhibitory weight increase for interneurons recruited by the losing group (bottom). (C) Effect of inhibition on the response of the postsynaptic neuron in response to correlated events for $τ_{in} = 2.12$ ms (left) and 3.54 ms (right). Comparison of the signal/noise ratio(SNR) between specific inhibition (red), the control of unspecific inhibition (green) and excitation only (black). (D) $τ_{in}$ and $τ_{out}$ results for the Noisy FFC with specific inhibition (red), unspecific inhibition (green), and with only excitation (black) for various $τ_{in}$.

**Supplementary Figure 3 | Rebound of the neuronal spike probability after the arrival of strong inhibition in the SFC.** The rebound in spiking probability is visible for specific inhibition (red curves). In the specific inhibition case, a rebound response is observed (orange arrow). The red arrow indicates the moment inhibition kicks in, in the specific inhibition case. The rebound response is shown for $τ_{in} = 0.71$ ms, $d = 5$, and 6 ms.
REFERENCES

Anderson, J., Carandini, M., and Ferster, D. (2000). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex. *J. Physiol.* 84, 909–926.

Atallah, B., and Scanziani, M. (2009). Instantaneous modulation of gamma oscillation frequency by balancing excitation with inhibition. *Neuron* 62, 566–577. doi: 10.1016/j.neuron.2009.04.027

Bell, C., Han, V., Sugawara, Y., and Grant, K. (1997). Synaptic plasticity in a cerebellum-like structure depends on temporal order. *Nature* 246, 170–170.

Bi, G., and Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. *J. Neurosci.* 18, 10464–10472.

Böhme, G., Bon, C., Lemaire, M., Reibaud, M., Piot, O., Stutzmann, J., et al. (1993). Altered synaptic plasticity and memory formation in nitric oxide synthase inhibitor-treated rats. *Proc. Natl. Acad. Sci. U.S.A.* 90, 9191–9194. doi: 10.1073/pnas.90.19.9191

Bourjaily, M. A., and Miller, P. (2011a). Excitatory, inhibitory, and structural plasticity produce correlated connectivity in random networks trained to solve paired-stimulus tasks. *Front. Comput. Neurosci.* 5:37. doi: 10.3389/fncom.2011.00037

Bourjaily, M. A., and Miller, P. (2011b). Synaptic plasticity and connectivity requirements to produce stimulus-pair specific responses in recurrent networks of spiking neurons. *PLoS Comput. Biol.* 7:e100109. doi: 10.1371/journal.pcbi.1001091

Brette, R. (2008). Generation of correlated spike trains. *Neural Comput.* 21, 188–215. doi: 10.1162/neco.2009.12-07-657

Buzsaki, G. (1984). Feed-forward inhibition in the hippocampal formation. *Prog. Neurobiol.* 22, 131–153. doi: 10.1016/0301-0082(84)90023-6

Carnell, A. (2009). An analysis of the use of Hebbian and Anti-Hebbian spike time dependent plasticity learning functions within the context of recurrent spiking neural networks. *Neurocomputing* 72, 685–692. doi: 10.1016/j.neucom.2008.07.012

Clopath, C., Büsing, L., Vasilaki, E., and Gerstner, W. (2010). Connectivity reflects coding: a model of voltage-based STDP with homeostasis. *Nat. Neurosci.* 13, 344–352. doi: 10.1038/nn.2479

Davis, K., Miller, R., and Young, E. (1996). Effects of somatosensory and parallel-fiber stimulation on neurons in dorsal cochlear nucleus. *J. Neurophysiol.* 76, 3012–3024.

Debanne, D., Beat, H., and Thompson, S. (1998). Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. *J. Physiol.* 507, 237–248. doi: 10.1111/j.1469-7793.1998.237bu.x

Diesmann, M., Gewaltig, M., and Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. *Nature* 402, 529–533. doi: 10.1038/990101

Fino, E., Paille, V., Deniau, J., and Venance, L. (2009). Asymmetric spike-timing dependent plasticity of striatal nitric oxide-synthase interneurons. *Neuroscience* 160, 744–754. doi: 10.1016/j.neuroscience.2009.03.015

Gabernet, L., Jadhav, S., Feldman, D., Carandini, M., and Scanziani, M. (2005). Somatosensory integration controlled by dynamic thalamocortical feed-forward inhibition. *Neuron* 48, 315–327. doi: 10.1016/j.neuron.2005.09.022

Gerstner, W., Kempter, R., van Hemmen, J., and Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. *Nature* 383, 76–68. doi: 10.1038/383076a0

Gilson, M., Burkitt, A., and van Hemmen, L. (2010). STDP in recurrent neuronal networks. *Front. Comput. Neurosci.* 4:23. doi: 10.3389/fncom.2010.00023

Gilson, M., and Fukai, T. (2011). Stability versus neuronal specialization for STDP: long-tail weight distributions solve the dilemma. *PLoS ONE* 6:e25339. doi: 10.1371/journal.pone.0025339

Gjorgjieva, J., Clopath, C., Audet, J., and Pfister, J. (2011). A triplet spike-timing-dependent plasticity model generalizes the Bienenstock-Cooper-Munro rule to higher-order spatiotemporal correlations. *Proc. Natl. Acad. Sci. U.S.A.* 108, 19383–19388. doi: 10.1073/pnas.1105933108

Goodman, D., and Brette, R. (2008). Brian: a simulator for spiking neural networks in python. *Front. Neuroinform.* 2:5. doi: 10.3389/neuro.11.005.2008

Gütig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. *J. Neurosci.* 23, 3697–714.

Haas, J., Nowotny, T., Abarbanel, H., Zavala, B., and Landisman, C. (2006). Spike-timing-dependent plasticity of inhibitory synapses in the entorhinal cortex. *J. Neurophysiol.* 96, 3305–3313. doi: 10.1152/jn.00551.2006

Haider, B., Duque, A., Hasenstaub, A., and McCormick, D. (2006). Neocortical network activity *in vivo* is generated through a dynamic balance of excitation and inhibition. *J. Neurosci.* 26, 4535–4545. doi: 10.1523/JNEUROSCI.5297-05.2006

Haider, B., Häusser, M., and Carandini, M. (2013). Inhibition dominates sensory responses in the awake cortex. *Nature* 493, 97–100. doi: 10.1038/nature11665

Han, V., Grant, K., and Bell, C. (2000). Reversible associative depression and nonassociative potentiation at a parallel fiber synapse. *Neuron* 27, 611–622. doi: 10.1016/S0896-6273(00)00070-2

Harvey-Girard, E., Lewis, J., and Maler, L. (2010). Burst-induced anti-Hebbian depression acts through short-term synaptic dynamics to cancel redundant sensory signals. *J. Neurosci.* 30, 6152–6169. doi: 10.1523/JNEUROSCI.0303-10.2010

Hensch, T., Fagiolini, M., Mataga, N., Stryker, M., Baekkeskov, S., and Kash, S. (1998). Local GABA circuit control of experience-dependent plasticity in developing visual cortex. *Science* 282, 1504–1508. doi: 10.1126/science.282.5393.1504

Hong, S., Ratté, S., Prescott, S., and De Schutter, E. (2012). Single neuron firing properties impact correlation-based population coding. *J. Neurosci.* 32, 1413–1428. doi: 10.1523/JNEUROSCI.3735-11.2012

Izhikevich, E., Gally, J., and Edelman, G. (2004). Spike-timing dynamics of neuronal groups. *Cereb. Cortex* 14, 933–944. doi: 10.1093/cercor/bhh053

Jackson, A., Gee, V., Baker, S., and Lemon, R. (2003). Synchrony between neurons with similar muscle fields in monkey motor cortex. *Neuron* 38, 115–125. doi: 10.1016/S0896-6273(03)00162-4

Kang, S., Kitano, K., and Fukai, T. (2008). Structure of spontaneous UP and DOWN transitions self-organizing in a cortical network model. *PLoS Comput. Biol.* 4:e1000022. doi: 10.1371/journal.pcbi.1000022

Kempter, R., Gerstner, W., and van Hemmen, J. (1999). Hebbian learning and spiking neurons. *Phys. Rev. E* 59, 4498–4514. doi: 10.1103/PhysRevE.59.4498

Kilavik, B., Confais, J., Gruen, S., and Riehle, A. (2009). Long-term modifications in motor cortical dynamics induced by intensive practice. *J. Neurosci.* 29, 12653–12663. doi: 10.1523/JNEUROSCI.1554-09.2009

Kodangattil, J. N., Dacher, M., Authement, M. E., and Nugent, F. S. (2013). Spike timing-dependent plasticity at GABAergic synapses in the ventral tegmental area. *J. Physiol.* 591, 4699–4710. doi: 10.1113/jphysiol.2013.257873

Kremkow, J., Aertsen, A., and Kumar, A. (2010). Gating of signal propagation in spiking neural networks by balanced and correlated excitation and inhibition. *J. Neurosci.* 30, 15760–15768. doi: 10.1523/JNEUROSCI.3874-10.2010

Luz, Y., and Shamir, M. (2012). Balancing feed-forward excitation and inhibition via Hebbian inhibitory synaptic plasticity. *PLoS Comput. Biol.* 8:e1002334. doi: 10.1371/journal.pcbi.1002334

Maffei, A., Nataraj, K., Nelson, S., and Turrigiano, G. (2006). Potentiation of cortical inhibition by visual deprivation. *Nature* 443, 81–84. doi: 10.1038/nature05079

Magee, J., and Johnston, D. (1997). A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. *Science* 275, 209–213. doi: 10.1126/science.275.5297.209

Maldonado, P., Babul, C., Singer, W., Rodriguez, E., Berger, D., and Grün, S. (2008). Synchronization of neuronal responses in primary visual cortex of monkeys viewing natural images. *J. Neurophysiol.* 100, 1523–1532. doi: 10.1152/jn.00076.2008

Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. *Science* 275, 213–215. doi: 10.1126/science.275.5297.213

Markram, H., Toledo-rodriguez, M., Wang, Y., and Gupta, A. (2005). Interneurons of the neocortical inhibitory system. *Nat. Rev. Neurosci.* 5, 793–807. doi: 10.1038/nrn1519

Okun, M., and Lampl, I. (2008). Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities. *Nat. Neurosci.* 11, 535–537. doi: 10.1038/nn.2105

Pouille, F., Marin-Burgin, A., Adesnik, H., Atallah, B., and Scanziani, M. (2009). Input normalization by global feedforward inhibition expands cortical dynamic range. *Nat. Neurosci.* 12, 1577–1585. doi: 10.1038/nn.2441

Pouille, F., and Scanziani, M. (2001). Enforcement of temporal fidelity in pyramidal cells by somatic feed-forward inhibition. *Science* 293, 1159–1163. doi: 10.1126/science.1060342
Putrino, D., Brown, E., Mastaglia, F., and Ghosh, S. (2010). Differential involve-
ment of excitatory and inhibitory neurons of cat motor cortex in coincident
spike activity related to behavioral context. *J. Neurosci.* 30, 8048–8056. doi:
10.1523/JNEUROSCI.0770-10.2010

Riehle, A., Gruen, S., Diesmann, M., and Aertsen, A. (1997). Spike synchronization and rate modulation differentially involved in motor cortical function. *Science* 278, 1950–1953. doi: 10.1126/science.278.5345.1950

Roberts, P., and Bell, C. (2000). Computational consequences of temporally asym-
metric learning rules: II. Sensory image cancellation. *J. Comput. Neurosci.* 9,
67–83. doi: 10.1023/A:1008938428112

Rossant, C., Leijon, S., Magnusson, A., and Brette, R. (2011). Sensitivity of noisy neurons to coincident inputs. *J. Neurosci.* 31, 17193–1206. doi: 10.1523/JNEUROSCI.2482-11.2011

Rudolph, M., Pospischil, M., Timofeev, I., and Destexhe, A. (2007). Inhibition determines membrane potential dynamics and controls action potential gen-eration in awake and sleeping cat cortex. *J. Neurosci.* 27, 5280–5290. doi:10.1523/JNEUROSCI.4652-06.2007

Rumsey, C., and Abbott, L. (2004). Equalization of synaptic efficacy by activity-
and timing-dependent synaptic plasticity. *J. Neurophysiol.* 91, 2273–2280. doi:
10.1152/jn.00900.2003

Rumsey, C., and Abbott, L. (2006). Synaptic democracy in active dendrites. *J. Neurophysiol.* 96, 2307–2318. doi: 10.1152/jn.00149.2006

Savin, C., Joshi, P., and Triesch, J. (2010). Independent component analysis in spik-
ing neurons. *PLoS Comput. Biol.* 6:e1000757. doi: 10.1371/journal.pcbi.1000757

Senn, W., and Buchs, N. (2003). Spike-based synaptic plasticity and the emergence of direction selective simple cells: mathematical analysis. *J. Comput. Neurosci.* 14, 119–138. doi: 10.1023/A:1021935100586

Silberberg, G., and Markram, H. (2007). Disynaptic inhibition between neocor-
tical pyramidal cells mediated by Martinotti cells. *Neuron* 53, 735–746. doi:
10.1016/j.neuron.2007.02.012

Sjöström, P., Turrigiano, G., and Nelson, S. (2001). Rate, timing, and coopera-
tivity jointly determine cortical synaptic plasticity. *Neuron* 32, 1149–1164. doi:
10.1016/S0896-6273(01)00542-6

Song, S., and Abbott, L. (2001). Cortical development and remapping through
spike timing-dependent plasticity. *Neuron* 32, 339–350. doi: 10.1016/S0896-
6273(01)00451-2

Song, S., Miller, K., and Abbott, L. (2000). Competitive Hebbian learning through
spike-timing-dependent synaptic plasticity. *Nat. Neurosci.* 3, 919–926. doi:
10.1038/78829

Stokes, C., and Isaacson, J. (2010). From dendrite to soma: dynamic routing of inhi-
bition by complementary interneuron microcircuits in olfactory cortex. *Neuron*
67, 452–465. doi: 10.1016/j.neuron.2010.06.029

Tan, Z., Hu, H., Huang, Z., and Agmon, A. (2008). Robust but delayed thalamocor-
tical activation of dendritic-targeting inhibitory interneurons. *Proc. Natl. Acad.
Sci. U.S.A.* 105, 2187–2192. doi: 10.1073/pnas.0710628105

Tzounopoulos, T., Kim, Y., Oertel, D., and Trussell, L. (2004). Cell-specific, spike
timing-dependent plasticities in the dorsal cochlear nucleus. *Nat. Neurosci.* 7,
719–725. doi: 10.1038/nn1272

van Rossum, M., Bi, G., and Turrigiano, G. (2000). Stable Hebbian learning from
spike timing-dependent plasticity. *J. Neurosci.* 20, 8812–8821.

van Vreeswijk, C., and Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. *Science* 274, 1724–1726. doi: 10.1126/science.274.5293.1724

Vogels, T., and Abbott, L. (2009). Gating multiple signals through detailed balance of excitation and inhibition in spiking networks. *Nat. Neurosci.* 12, 483–491. doi: 10.1038/nn.2276

Vogels, T., Sprekeler, H., Zenke, F., Clopath, C., and Gerstner, W. (2011). Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. *Science* 334, 1569–1573. doi: 10.1126/science.1211095

Wang, L., and Maffei, A. (2014). Inhibitory plasticity dictates the sign of plasticity at excitatory synapses. *J. Neurosci.* 34, 1083–1093. doi: 10.1523/JNEUROSCI.4711-13.2014

Wehr, M., and Zador, A. (2003). Balanced inhibition underlies tuning and sharpens
spike timing in auditory cortex. *Nature* 426, 442–446. doi: 10.1038/nature02116

Wilent, W., and Contreras, D. (2004). Synaptic responses to whisker deflections in rat barrel cortex as a function of cortical layer and stimulus intensity. *J. Neurosci.* 24, 3985–3998. doi: 10.1523/JNEUROSCI.5782-03.2004

Woodin, M., Ganguly, K., and Poo, M. (2003). Coincident pre- and post-
synaptic activity modifies GABAergic synapses by postsynaptic changes in
Cl- transporter activity. *Neuron* 39, 807–820. doi: 10.1016/S0896-6273(03) 00507-5

Wu, G., Arbuckle, R., Liu, B., Tao, H., and Zhang, L. (2008). Lateral sharpening of cortical frequency tuning by approximately balanced inhibition. *Neuron* 58, 132–143. doi: 10.1016/j.neuron.2008.01.035

Wu, G., Li, P., Tao, H., and Zhang, L. (2006). Nonmonotonic synaptic excita-
tion and imbalanced inhibition underlying cortical intensity tuning. *Neuron* 52,
705–715. doi: 10.1016/j.neuron.2006.10.009

Young, J., Waleszczyk, W., Wang, C., Calford, M., Dreher, B., and Obermayer, K. (2007). Cortical reorganization consistent with spike timing-but not correlation-dependent plasticity. *Nat. Neurosci.* 10, 887–895. doi: 10.1038/nn0807-1073c

Zhang, L., Tao, H., Holt, C., Harris, W., and Poo, M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. *Nature* 395, 37–44. doi: 10.1038/25665

**Conflict of Interest Statement:** The authors declare that the research was con-
ducted in the absence of any commercial or financial relationships that could be
construed as a potential conflict of interest.

Received: 25 December 2013; accepted: 10 April 2014; published online: 07 May 2014.

Citation: Kleberg FI, Fukai T and Gilson M (2014) Excitatory and inhibitory STDP jointly tune feedforward neural circuits to selectively propagate correlated spiking activity. *Front. Comput. Neurosci.* 8:53. doi: 10.3389/fncom.2014.00053

This article was submitted to the journal Frontiers in Computational Neuroscience.

Copyright © 2014 Kleberg, Fukai and Gilson. This is an open-access article dis-
tributed under the terms of the Creative Commons Attribution License (CC BY). The
use, distribution or reproduction in other forums is permitted, provided the original
author(s) or licensor are credited and that the original publication in this jour-
nal is cited, in accordance with accepted academic practice. No use, distribution or
reproduction is permitted which does not comply with these terms.