# Emergence and maintenance of modularity in neural networks with Hebbian and anti-Hebbian inhibitory STDP

Raphaël Bergoin, Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López

<figure><img src="image_1.png" /></figure>

## To cite this version:

Raphaël Bergoin, Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López. Emergence and maintenance of modularity in neural networks with Hebbian and anti-Hebbian inhibitory STDP. PLoS Computational Biology, 2025, 21 (4), pp.e1012973. 10.1371/journal.pcbi.1012973. hal-05160250

HAL Id: hal-05160250

https://hal.science/hal-05160250v1

Submitted on 13 Jul 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
RESEARCH ARTICLE

# Emergence and maintenance of modularity in neural networks with Hebbian and anti-Hebbian inhibitory STDP

Raphaël Bergoin<sup>ID 1,2,3,4*</sup>, Alessandro Torcini<sup>5</sup>, Gustavo Deco<sup>2,3,6</sup>, Mathias Quoy<sup>1,7</sup>, Gorka Zamora-López<sup>2,3</sup>

1 ETIS, UMR 8051, ENSEA, CY Cergy Paris Université, CNRS, Cergy-Pontoise, France, 2 Center for Brain and Cognition, Pompeu Fabra University, Barcelona, Spain, 3 Department of Information and Communication Technologies, Pompeu Fabra University, Barcelona, Spain, 4 Institute of Neural Information Processing, Center for Molecular Neurobiology (ZMNH), University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany, 5 Laboratoire de Physique Théorique et Modélisation, UMR 8089, CY Cergy Paris Université, CNRS, Cergy-Pontoise, France, 6 Instituciò Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Spain, 7 IPAL, CNRS, Singapore, Singapore

* raphael.bergoin@gmail.com

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

OPEN ACCESS

**Citation:** Bergoin R, Torcini A, Deco G, Quoy M, Zamora-López G (2025) Emergence and maintenance of modularity in neural networks with Hebbian and anti-Hebbian inhibitory STDP. PLoS Comput Biol 21(4): e1012973. https://doi.org/10.1371/journal.pcbi.1012973

**Editor:** Alex Roxin, CRM: Centre de Recerca Matematica, SPAIN

**Received:** July 10, 2024

**Accepted:** March 19, 2025

**Published:** April 22, 2025

**Copyright:** © 2025 Bergoin et al. This is an open access article distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data availability statement:** All code written in support of this publication is publicly available at https://github.com/rbergoin/QIF-neurons-with-3-synaptic-plasticities.

**Funding:** This work was supported (RB, GD and GZL) by the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific [Grant Agreement No. 945539 (Human Brain Project SGA3)] and by an EUTOPIA funding [EUTOPIA-PhD-2020-0000000066 - NEUROAI].

## Abstract

The modular and hierarchical organization of the brain is believed to support the coexistence of segregated (specialization) and integrated (binding) information processes. A relevant question is yet to understand how such architecture naturally emerges and is sustained over time, given the plastic nature of the brain's wiring. Following evidences that the sensory cortices organize into assemblies under selective stimuli, it has been shown that stable neuronal assemblies can emerge due to targeted stimulation, embedding various forms of synaptic plasticity in presence of homeostatic and/or control mechanisms. Here, we show that simple spike-timing-dependent plasticity (STDP) rules, based only on pre- and post-synaptic spike times, can also lead to the stable encoding of memories in the absence of any control mechanism. We develop a model of spiking neurons, trained by stimuli targeting different sub-populations. The model satisfies some biologically plausible features: (i) it contains excitatory and inhibitory neurons with Hebbian and anti-Hebbian STDP; (ii) neither the neuronal activity nor the synaptic weights are frozen after the learning phase. Instead, the neurons are allowed to fire spontaneously while synaptic plasticity remains active. We find that only the combination of two inhibitory STDP sub-populations allows for the formation of stable modules in the network, with each sub-population playing a distinctive role. The Hebbian sub-population controls for the firing activity, while the anti-Hebbian neurons promote pattern selectivity. After the learning phase, the network settles into an asynchronous irregular resting-state. This post-learning activity is associated with spontaneous memory recalls which turn out to be fundamental for the long-term consolidation of the learned memories. Due to its simplicity, the introduced model can represent a test-bed for further investigations on the role played by STDP on memory storing and maintenance.
AT received financial support by the Labex
MME-DII [Grant No. ANR-11-LBX-0023-01]
(together with MQ), by the ANR Project
ERMUNDY [Grant No. ANR-18-CE37-0014] and
by CY Generations (Grant No.
ANR-21-EXES-0008) all part of the French
program "Investissements d'Avenir." GD is
supported by Grant PID2022-136216NB-I00
funded by MICIU/AEI/10.13039/501100011033
and by "ERDF A way of making Europe," ERDF.
MQ is also partially supported by CNRS through
the IPAL lab in Singapore. The funders had no
role in study design, data collection and
analysis, decision to publish, or preparation of
the manuscript.

**Competing interests:** The authors have
declared that no competing interests exist.

Author summary

One of the most remarkable qualities of the brain is its capacity to learn and adapt. How the learning process imprints and maintains memories, by shaping the architecture of connectivity among neurons in a constantly changing and dynamic environment, is a major question of neuroscience. Here, we explore the idea that the segregation of inputs received by a neural network, with inputs targeting distinct populations, is a key factor for shaping the architecture of the network. We find that the presence of inhibitory neurons is necessary for the emergence and the long-term maintenance of modularity in spiking neural networks with plasticity. In particular, we show that two different inhibitory sub-populations, one subject to Hebbian and the other to anti-Hebbian plasticity, are required to promote the formation of feedback and feed-forward inhibition circuits controlling memory consolidation. On one side, these inhibitory circuits favour long-term memory consolidation by inducing spontaneous memory recalls in the asynchronous irregular resting phase. On another side, the number of inhibitory neurons controls the maximal memory capacity of the considered model.

Introduction

The brain's connectivity follows a modular and a hierarchical organization at different spa-
tial and functional scales [1–6]. From an operative point of view, this type of architecture is
suggested to facilitate the coexistence of segregation and integration of information [7–9]:
neuronal circuits or brain regions associated to a specific function are densely connected with
each other [10–12], while long-range connections and network hubs allow for the integration
(or binding) of different information [9,13,14]. A crucial open question in brain connectiv-
ity is to understand how such modular and hierarchical organization naturally emerges as a
consequence of the functional needs of the nervous system.

This question has been addressed from various standpoints. On the one hand, it has
been shown that spontaneous neural activity might be sufficient to shape network struc-
ture [15–18]. The underlying idea is that neural dynamics split brain connectivity into dif-
ferent populations via Hebbian-like adaptation mechanisms that reinforce links between
partially synchronised neurons. Such mechanisms may take place, for example, during early
development when cortical areas become organized in the absence of sensory stimuli [19].
The resulting complex connectivity—in turn—supports a combination of time-scales and this
facilitates the onset of self-sustained metastable neural activity [15,20], e.g., the spontaneous
switching of activity across neural populations [21,22].

On the other hand, the relation between learning and network organization is inherently
associated with the notions of semantic memory and input selectivity. Working memory
is characterised by synaptic changes leading to groups of neurons (assemblies or engrams)
to sustain higher frequencies for a few seconds after stimulus presentation [23,24]. These
changes also facilitate recognition if the stimulus is presented again shortly after. Sensory
cortices contain neurons selectively firing for different features of the inputs, forming dif-
ferentiated groups of neurons (assemblies) related to receptive fields [25]. Following these
observations, computational models of spiking neurons have been proposed to inves-
tigate how memories could be imprinted into the neuronal architecture [17,21,26–38].
Starting from a random connectivity, their goal is to reproduce the formation of neuronal
assemblies in response to various external stimuli (or memories), mediated by synaptic
plasticity.
This process comes with several challenges and questions:

(i) the correct formation of neural assemblies (modules) in the network architecture that are consistent with the learned stimuli,

(ii) the long-term stability of these structures when the entrainment is finished, and

(iii) the stability of the resulting network dynamics.

The mechanism for long-term memory maintenance and consolidation has been related to spontaneous memory recalls (or retrievals). During wake, short and random events of partial synchronous activation of neural sub-networks are related both to the spontaneous recalls [39] and to the consolidation [40–42] of learned memories. Another important aspect is then to clarify how the typical cortical dynamics, that is asynchronous and irregular, can coexist with these spontaneous recalls [43,44].

A crucial aspect for the success of the proposed neural networks is the selection of a model for the plasticity. The exact mechanisms underlying synaptic plasticity have been long debated but the pioneering experiments by Bi & Poo [45,46] on the hippocampus revealed that the plastic changes are driven by a temporally asymmetric form of Hebbian learning, induced by tight temporal correlations between the spikes of pre- and postsynaptic neurons. This phenomenon has been termed as spike-timing-dependent plasticity (STDP) [47–49]. However, previous attempts to model the storage of memory items via stimulations of selected neurons in random balanced networks have led to unstable behaviours impeding the formation of stimulus induced structures [29]. To avoid these pathological behaviours other models have been proposed which, instead, combine different types of voltage- and rate-dependent STDP, often in presence of homeostatic or other control mechanisms to prevent instabilities. In particular, in [31,38] neuronal assemblies were successfully formed which reflected the stored memory items, and their long-term maintenance was mediated by spontaneous recalls of the stored patterns. The main difference among these two studies is how the STDP rules were implemented. In Litwin-Kumar and Doiron [31] a voltage-based STDP rule [30] for the excitatory-excitatory synapses was employed to achieve a stable evolution, together with additional homeostatic mechanisms based on inhibitory plasticity and on non-local renormalization of the excitatory-excitatory synaptic weights. In Yang & La Camera [38] plastic synapses were considered only among excitatory neurons and no further homeostatic mechanisms. The STDP rule is similar to the one in Ref. [31] where a voltage-sensitive variable—depending on the post-synaptic membrane potential—is compared to a threshold in order to determine whether the synapse enters a long-term potentiation or depression state. The main difference is that the thresholds in Yang & La Camera [38] evolve dynamically as in the BCM rule [50], adapting to the single neuron post-synaptic activity in order to avoid instabilities in the dynamics.

In the present paper, we consider simpler STDP mechanisms based only on the local information associated with pre- and post-synaptic spike times analogous to the one discovered by Bi and Poo [45,46]. Contrary to the results in Morrison et al. [29] our network model successfully encodes and sustains stimulus-driven assemblies in the absence of any additional control mechanism. This is achieved by accounting for inhibitory STDP mechanisms alongside the excitatory one. Recent empirical studies have underlined the variety of functional roles played by different classes of interneurons with, for example, parvalbumin-expressing (PV) interneurons in the mouse being subject to symmetric Hebbian STDP and somatostatin-expressing (SOM) ones following asymmetric Hebbian STDP [42,51,52]. Accordingly, our model considers two inhibitory neuronal sub-populations which self-organise during the training phase into two differential functional roles: one sub-population subjected to Hebbian STDP (which
controls the firing activity) and another population following anti-Hebbian STDP (which
mediates for memory selectivity).

After the learning phase the model settles into an asynchronous irregular state, as typically
observed during *in-vivo* recordings of the brain activity at rest [53]. This post-learning activ-
ity is characterized by the occurrence of transient events of partial synchrony, associated to
the learned items as in [31–33,38]. These spontaneous memory recalls are crucial for the long-
term consolidation of the stored memories by promoting the reinforcement of the underlying
connectivity. Given that inhibitory plasticity takes an active role in the model, the memory
capacity of the network is controlled by the number of inhibitory neurons. Finally, we also
demonstrate the emergence of hub neurons displaying mixed selectivity, by training the net-
work to overlapping memory items [17,34,36–38]. These excitatory hub neurons represent the
seeds for hierarchical organization and integration.

Results

We consider networks of excitatory and inhibitory Quadratic Integrate-and-Fire (QIF) [54] neurons, pulse coupled via exponentially decaying post-synaptic potentials and in the presence of STDP. The connections involving pre-synaptic excitatory neurons are subject to Hebbian STDP, while those involving pre-synaptic inhibitory neurons can be subject to two types of STDP: either Hebbian or anti-Hebbian.

Firstly, we will investigate the necessary conditions for the emergence of modular assem-
blies induced by learning selective stimuli. For this purpose, we will analyze the role played by
Hebbian and anti-Hebbian plasticity applied to inhibitory neurons in a network subject to two
external stimuli. Secondly, we will investigate the role of spontaneous recalls in order to con-
solidate and maintain both the learned memory items and the underlying modular connectiv-
ity. To clarify this aspect we will perturb the synaptic connectivity matrix and we will exam-
ine to which extent spontaneous recalls are able to regenerate the original structure induced
by the training. The robustness of this scenario will be validated by considering larger system
sizes and random networks. Finally, we will generalize the model to account for an increasing
number of stored memories and in the presence of overlapping assemblies, thus showing that
more complex architectures can develop and be maintained over time.

Emergence of modular connectivity driven by learning selective stimuli

We begin by investigating the necessary conditions for the emergence of modular structure as
induced by training with external stimuli. As a general set-up, we consider heterogenous glob-
ally coupled networks composed of N = 100 QIF neurons, each neuron receiving weak and
independent Gaussian noise. The neurons with labels in the interval i ∈ [0 : 79] are excitatory
and those labelled i ∈ [80 : 99] are inhibitory. We consider the neurons to be non-identical
with their excitabilities normally distributed, leading to spontaneous firing frequencies in
the range [0,8] Hz. Synaptic weights wᵢⱼ from the j-th pre-synaptic neuron to the i-th post-
synaptic one are subject to STDP, with weights bounded in the interval wᵢⱼ ∈ [0, 1] for exci-
tatory pre-synaptic neurons and in the interval wᵢⱼ ∈ [-1, 0] for inhibitory ones. See Methods
for more details.

The stimulation protocol consists of three stages, as illustrated in Fig 1A : an initial resting phase followed by a learning period and a final consolidation phase. The simulations are performed by ensuring the model respects some biologically realistic conditions: (i) the networks are allowed to continue their spontaneous activity after the learning phase; (ii) the adaptation of the synaptic weights is always active throughout the whole simulation, i.e., before, during and after the learning.
<figure><img src="image_5.png" /><figcaption>**B** Anti-Hebbian inhibition only</figcaption></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /><figcaption>**C** Hebbian inhibition only</figcaption></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /><figcaption>**D** Anti-Hebbian and Hebbian inhibition</figcaption></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>**Fig 1.** Emergence of modular connectivity by learning two selective stimuli in spiking networks. (A) Experimental protocol consists of the stimulation of two non-overlapping neuronal populations of QIF neurons with plastic synapses. Networks</figcaption></figure>
are made of 80% of excitatory and 20% of inhibitory neurons. Stimuli are presented in temporal alternation. Results of the performed numerical experiments are reported for a network with all anti-Hebbian inhibitory neurons (B); with all Hebbian inhibitory neurons (C); with 50% anti-Hebbian and 50% Hebbian inhibitory neurons (D). Raster plots display the firing times of excitatory (red dots) and inhibitory (blue dots) neurons during the simulations. Matrices represent the temporal evolution of the connection weights at different times: *t* = 0s (random initialization of the weights), *t* = 20s (middle of the learning phase) and *t* = 40s (end of the learning phase). The color denotes if the connection is excitatory (red), inhibitory (blue) or absent (white) and the color gradation the strength of the synaptic weight. The final configuration of the connection weights is shown schematically on the right in each case.

https://doi.org/10.1371/journal.pcbi.1012973.g001

Initially, despite the network being globally coupled the weights are randomly assigned with small positive (negative) values for excitatory (inhibitory) synapses. The system is left to relax for five seconds in the absence of external stimuli. During this stage the network stabilizes into an asynchronous state with the neurons firing irregularly at low frequencies, as shown in the raster plots corresponding to the time interval [0 : 5] seconds in Fig 1. The emergence of the asynchronous irregular dynamics in our globally coupled model is due to the heterogenous distribution of the excitabilities and of the weights, and the external noise. However, a similar dynamics can be observed in networks of identical neurons (same excitabilities) with random connectivity, as shown in S6 Text and in [31,38,44].

During the learning phase two independent stimuli are applied, each targeting a different (non-overlapping) neuronal population, as shown in Fig 1A. This is done in order to mimic the segregation of (sensory) information being projected to nearby but separate neuronal populations, and to study the role of this segregation for the emergence of modular neuronal architectures. In particular, one stimulus targets population P₁ consisting of the first half of excitatory and inhibitory neurons labelled as E₁ := {i ∈ [0 : 39]} and I₁ := {i ∈ [80 : 89]}. The second stimulus targets population P₂ consisting of the second half of excitatory and inhibitory neurons, E₂ := {i ∈ [40 : 79]} and I₂ := {i ∈ [90 : 99]}.

The two stimuli are applied for a time duration of one second, randomly alternating between the two populations P₁ and P₂. During each stimulation period, a constant external positive current is applied to the selected population for 800 ms. The stimulus triggers the target neurons to fire at about 50 Hz. After 800 ms the external current is turned off and the network is left to relax for other 200 ms in order to prevent temporal correlations when alternating between stimulated populations. This protocol is repeated 35 times for a total of 35 seconds. Once the training phase is finished, the network is allowed to evolve freely for 20 seconds in the absence of stimuli. During this phase, however, the synaptic adaptation remains active thus affecting the stabilization of the learned patterns.

**Role of Hebbian and anti-Hebbian learning for the emergence of modular connectivity**

To explore the role played by the inhibitory neurons in the learning process, we study three different scenarios: (a) all inhibitory neurons follow anti-Hebbian STDP, (b) all inhibitory neurons follow Hebbian learning, and (c) a mixed situation where 50% are Hebbian and 50% anti-Hebbian.

(a) *Anti-Hebbian inhibition.* In the case with only anti-Hebbian inhibitory plasticity, the network develops into a *winner-takes-all* architecture promoting the competition of the two sub-populations P₁ or P₂ on the other one, as shown in Fig 1B. In order to describe the mechanism leading to this architecture, imagine that population P₁ is stimulated, consequently all its neurons become active and fire with high frequency. Therefore all the connections E₁ → {E₁, I₁} are reinforced due to the Hebbian nature of the pre-synaptic excitatory neurons. At the same time, all the inhibitory synapses I₁ → {E₁, I₁} are weakened because they are anti-Hebbian. During the stimulation of P₁, the neurons
in populations P₁ and P₂ are far from being mutually synchronized, therefore the con-
nections E₁ → {E₂, I₂} are weakened while I₁ → {E₂, I₂} reinforce. The random alterna-
tion of the stimuli to populations P₁ and P₂ induces a gradual emergence of a modular
structure, as visible in the connectivity (weight) matrices at times t = 20 sec and t = 40
sec in Fig 1B.

The resulting architecture promotes the competition between the two excitatory populations E₁ and E₂ and the alternating prevalence of one of them. For example, stimulation to E₂ would activate the companion inhibitory neurons through the *feed-forward* connections (E₂ → I₂) which, in turn, would shut down all neurons in P₁ via the strong I₂ → {E₁, I₁} connections. This is visible in the raster plot in Fig 1B : as the training progresses (t = 5 – 40 sec), the neurons of the stimulated population fire at high frequency but the neurons of the non-stimulated one are silenced.

(b) *Hebbian inhibition*. In the presence of only Hebbian inhibitory neurons the network also develops a modular organization, however in this case the two populations become disconnected from each other, Fig 1C. Analogously to the previous case, the stimulation of one population (e.g., P₁) results in the strengthening of the intra excitatory connections (increase of E₁ → {E₁, I₁} synaptic weights) and the weakening of the inter excitatory connections across different populations (decrease of E₁ → {E₂, I₂} weights). However, the Hebbian inhibitory plasticity now induces the strengthening of the internal inhibitory connections I₁ → {E₁, I₁} and the weakening of the inhibitory synapses across populations. The disconnection of the two populations happens gradually during the learning phase. As shown in the raster plot in Fig 1C, during the initial stimulation epochs, the stimulated population shuts down the activity of the non-stimulated population. But as the training proceeds, the two populations detach one from the other and the non-stimulated population begins to display a low firing resting-state activity.

(c) *Mixed inhibition*. In the last case with mixed anti-Hebbian and Hebbian inhibitory neurons, the network also develops two modules, but, as shown in Fig 1D, the resulting connectivity is a combination of the two configurations observed in the previous cases. The Hebbian neurons form self E-I loops within each population (i.e. E₁ − I₁H and E₂ − I₂H). This internal feedback inhibition avoids that the excitatory neurons fire at too large rates. Meanwhile, the anti-Hebbian inhibitory neurons form lateral, feed-forward connections which shut down the firing of the other population, in other terms the sub-population I₁A (I₂A) inhibits all neurons in P₂ (P₁).

**Resting-state network dynamics after learning** So far, we have shown that selective stim-
ulation to distinct populations consistently gives rise to modular networks and that the result-
ing configuration depends on the type of plasticity affecting the inhibitory neurons. However,
the relevance or plausibility of a learning model should also require that the network exhibits
a biologically meaningful dynamical behaviour after training [27,28,31–34,38].

(a) *Anti-Hebbian inhibition*. In the simulations with anti-Hebbian inhibitory STDP, the neuronal activity in the post-learning stage (*t*>40 sec) is dominated by one of the two populations. In Fig 1B, *P*₂ is active and *P*₁ is silent. But this randomly changes over realizations. In general, once the training stage is finished, all neurons start to fire spontaneously driven by the background Gaussian noise. The excitatory sub-population that attains a larger level of internal activity earlier is the one that wins. Due to the lack of internal feedback inhibition (*I*₁ ↔ *E*₁ or *I*₂ ↔ *E*₂) the winning excitatory sub-population fires rapidly and the companion inhibitory neurons (strongly stimulated) suppress the
activity of the other population: e.g. if the neurons in $E_2$ fire faster than those in $E_1$ the sub-population $I_2$ inhibits both $E_1$ and $I_1$.

(b) *Hebbian inhibition.* In the case with Hebbian inhibitory neurons, we found that the two populations $P_1$ and $P_2$ become independent, each forming a *feedback E-I loop*. The presence of the internal feedback inhibition allows the two populations to settle into a resting-state behaviour after training that is characterized by a low firing frequency of approximately 1.0 Hz, as visible in the raster plot of Fig 1C, for $t>40$ sec. Interestingly, brief events of internal partial synchrony of the two populations are also observed. While in the case with only anti-Hebbian inhibition such a synchronized event would trigger the excitatory neurons to permanently increase their firing, here the presence of the internal feedback inhibition ($I_1 \to E_1$ and $I_2 \to E_2$) avoids the constant synchronization of excitatory neurons, while it keeps their activity at low frequency.

(c) *Mixed inhibition.* The post-learning behaviour in the mixed scenario is very similar to the Hebbian case, as shown in Fig 1D. However, in this case populations $P_1$ and $P_2$ are not independent but they inhibit each other. As a consequence, an event of partial synchrony occurring in one of the two modules temporarily shuts down the other module, avoiding their mutual synchrony. Given that the plasticity remains active during this post-learning spontaneous activity, the connection weights continue to be updated. As we will discuss in the following, the occurrence of these spontaneous synchronization events plays a relevant role in the maintenance of the learned memories.

In summary, despite modular organization of the connectivity was found to emerge in the three considered scenarios, only the combination of anti-Hebbian and Hebbian inhibitory neurons resulted in a network satisfying all the desired biologically plausible properties. Anti-Hebbian inhibition alone led to a network with unrealistic post-learning dynamics. Hebbian inhibition alone gives rise to a biologically meaningful resting-state behaviour but at the cost of splitting the network into two disconnected populations. Therefore, in the following we will limit to consider the mixed scenario, with both anti-Hebbian and Hebbian inhibition.

The numerical experiments presented so far were repeated for a variety of special cases in order to validate the robustness of the results and the model: namely, (i) some of the neurons are not trained by stimuli (see S1 Text), (ii) at every stimulation epoch random subsets of neurons are targeted (see S2 Text), (iii) the intensity of the stimulation current is randomly fluctuating (see S3 Text), (iv) the network is composed of a large number of neurons, and (v) the neurons are randomly connected with an Erdös-Renyi distribution of the directed links (see S6 Text).

## Spontaneous recalls support consolidation and long-term maintenance of memories

Given that in our model plasticity is not frozen after training—but it is left active afterwards—we now investigate the potential role of the spontaneous events observed during the post-learning resting-state for the consolidation and maintenance of the memories. We will refer to these events as *memory recalls*. First, we will show that the recalls facilitate the completion of imperfectly learned memories and, second, we will study their role for the regeneration of memories that have been partially lost.

**Memory consolidation** In order to mimic a hypothetical scenario in which the training stage would stop before completion, we prepared an initial weight matrix representing an imperfectly learned connectivity structure. An example of this weight matrix is shown at $t = 0$ in Fig 2A. Specifically, the intra-modular synaptic weights are set to $w_E = 0.7$ for excitatory
<figure><img src="image_12.png" /><figcaption>**Fig 2.** Consolidation of imperfectly learned memories. (A) Temporal evolution of the connectivity matrix during spontaneous activity in the absence of stimulation. Initial connectivity with two unfinished modules at *t* = 0 is reinforced over time. The excitatory (inhibitory) connections are marked as in Fig 1. (B) Evolution of the distribution of link weights (probability density functions, PDFs, in a linear-logarithmic scale) in the connectivity matrices at *t* = 0s (light green), *t* = 400s (cyan) and *t* = 4000s (magenta). (C-J) Evolution of the network activity and various metrics in absence of stimulation for a sample of 30 seconds. Some spontaneous recalls are highlighted, for population P₁ (green shade) and P₂ (orange shade). Pink shadow marks an epoch of asynchronous irregular firing activity without recalls. (C) Raster plot with excitatory (inhibitory) neurons marked in red (blue). (D) PDFs of the coefficient of variations CVᵢ for all the neurons</figcaption></figure>
during the entire simulation (grey) and for a homogeneous Poisson process (yellow). (E) Instantaneous Kuramoto order parameters *R* for the networks (gray) and for neurons in population P₁ (green) and P₂ (orange), and their corresponding PDFs over the entire simulation (F). (G) Temporal evolution of the mean firing rates for populations P₁ and P₂, and (H) their corresponding frequency distributions (in linear-logarithmic scale) showing a peak at 2 Hz and long time tail. (I) Instantaneous change rates of synaptic weights in both populations P₁ and P₂ over time and their PDF (in linear-logarithmic scale) (J) over the entire simulation time, showing the prevalence of positive weight changes (reinforcement) with respect to negative ones (depression).

https://doi.org/10.1371/journal.pcbi.1012973.g002

and to $w_I = -0.7$ for inhibitory connections. The inter-modular excitatory (inhibitory) connections are chosen randomly from a Gaussian distribution with mean 0 and standard deviation 0.15 restricted to positive (negative) values.

Once fixed the initial weight matrix, the activity of the network is then left evolve spontaneously, driven only by the background Gaussian noise. However, we observe that the modular organization of the network is reinforced: the intra-modular synaptic weights of the two populations are strengthened and the inter-modular synapses are weakened, as shown by the weight matrices displayed in Fig 2A and the corresponding weight distributions in Fig 2B at $t = 400$ sec and $t = 4000$ sec.

The mechanism allowing for the completion of the connectivity pattern is summarized in Figs 2C-J. Panel C shows a 30 seconds sample of activity of the network. This is characterized by an asynchronous irregular evolution with low firing spontaneous activity joined to occasional spontaneous recalls occurring at random times. This observation is confirmed by the fact that the background activity is characterized by a distribution of the coefficient of variations $CV_i$ of the neurons in the interval [0.8 : 1.0] (panel D, grey distribution) lying close to a Poisson process (in yellow), thus resembling the irregular activity observed in the cortex *in vivo* [55]. The synchronization order parameter of the network *R* fluctuates in time with values around 0.2 (panel F, gray distribution), meaning that the network is far from being synchronized. As a matter of fact, for an asynchronous network of *N* = 100 neurons, due to finite size fluctuations the expected value of the order parameter will be not far from the one here measured, namely $R \approx 1/\sqrt{N} = 0.1$. Finally, the distribution of firing rates displays a main peak around 2 Hz and an exponential tail reaching at most 20 Hz (panels G and H).

During spontaneous recalls, the transient increase in synchrony and firing rates of a subset of the neurons (in one of the two clusters) triggers the activation of their synaptic adaptation (panel I), reshaping the synaptic weights and completing the formation of the modular patterns. The specific neurons participating in a recall changes from event to event but they tend to involve a majority of neurons of either population P₁ or P₂; see for example the events highlighted by green and orange shadows. The recalls coincide with transient peaks of the order parameter and of the mean firing rates of the populations involved (panels E and G, green lines). Consequently, P₁ and P₂ are internally more synchronized than the network average, with their order parameters fluctuating around 0.4 (panel F), and a broad distribution of firing rates (panel H). The similarity between the PDFs for populations P₁ and P₂ evidences that recalls occur—on average—with the same probability in both populations. Over time, the net effect of recalls is that the intra-modular synapses between neurons of the same population become reinforced while the inter-modular connections are weakened.

At this point, we remind that our model incorporates a natural, slow tendency to forget the value of the synaptic weights. Therefore, the question arises of how often should spontaneous recalls happen—and how strong should they be—in order to avoid the forgetting of the learned memories. In the example considered here, the weights between the neurons of a population undergo a natural depression during epochs of the resting-activity without recalls (pink shadow in Fig 2C). For the used parameters, a forgetting term of $f = 0.1$ and an average spontaneous firing rate of 2 Hz, we estimate that a period of $\approx 11$ seconds of asynchronous
firing would be required to forget the contribution of a single recall event (for more details on the calculus see S9 Text). The dominance of the reinforcement during recalls with respect to depression during the asynchronous irregular epochs is confirmed by the asymmetry in the distribution of positive weight change rates over negative ones, Fig 2J.

In summary, we can conclude that, in our model, spontaneous recalls happening during resting-state activity allow for the consolidation of imperfectly learned memories as well as for their long-term maintenance against natural forgetting [31,38].

**Regeneration of damaged excitatory and inhibitory connectivity** Empirical observations have shown that excitatory synapses are more volatile than inhibitory ones [56,57], leading to the hypothesis that reorganization of excitatory connections might be associated with short-term plasticity while inhibitory adaptation might support the long-term maintenance of those memories [58–61]. We now test the plausibility of this idea in our model.

We consider a first case where the weights of the excitatory synapses ($w_{EE}$ and $w_{IE}$) are initially randomized between 0 and 1, but the values of the weights of the inhibitory synapses coincide with those obtained as after the training stage. See the weight matrix for $t = 0$ in Fig 3A. The network is then let to evolve spontaneously for 24 hours, driven only by the background Gaussian noise. After one hour of simulated time, the excitatory connections have partially recovered the original modular pattern (see the weight matrix for $t = 1h$ in Fig 3A). This recovery is clearly mediated by the spontaneous recalls as illustrated in the raster plot in Fig 3A. Whenever a sufficiently large group of excitatory neurons (associated to one of the previously learned memories) fire in a short time window, this event activates the inhibitory neurons associated to the corresponding memory items and triggers a partial recall via the feedback and feed-forward mechanism previously explained. Consequently, the excitatory synapses involved in the recalled memory are partially reinforced while all other connections are weakened. The structure of the inhibitory weights appears to be deteriorating due to the more disordered activity observed in the network. As a result, in the long-term ($t = 12h$ and $t = 24h$), the modular connectivity structure encoding the stimuli is completely lost. Therefore, the maintenance of the inhibitory weight structure is sufficient for the (partial) recovery of the memory items over an intermediate time scale only.

The experiment was repeated but randomizing only the inhibitory weights ($w_{EI}$ and $w_{II}$, the corresponding weight matrix is shown at $t = 0$ in Fig 3B), the memorized pattern is fully recovered and maintained at all times. The main reason for this difference is related to the fact that when the excitatory synapses are preserved, the network displays a more sparse activity than with initially randomized excitatory connections. In the latter case, there are more interactions among excitatory neurons and this gives rise to a background activity involving most of the excitatory neurons. Therefore, one finally obtains a weight matrix with a more disordered structure (see in Fig 3A weight matrix at $t = 1h$), which becomes unstable after several hours of spontaneous activity. Another reason of this difference is related to the asymmetric 80–20% $E/I$ ratio, leading to notably different spontaneous firing frequencies—and recall probabilities—when either the excitatory or the inhibitory synapses are randomized.

Finally, the experiment is repeated by randomizing all weights except the excitatory to excitatory connections ($w_{EE}$), in Fig 3C. In the initial moments of the simulation, again we observe the spontaneous recall of excitatory neurons, but the inhibitory neurons due to the random distribution of the $E \to I$ connections are now randomly associated to one of the two clusters. This leads to a slightly different organization in the connectivity matrix than before, but with a similar architecture.

These results show that the conservation of the excitatory to excitatory $E \to E$ connections is indeed sufficient to maintain and reconsolidate the structure. Furthermore, our analysis
<figure><img src="image_13.png" /><figcaption>B I→E and I→I connections randomized</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>C E→I, I→E and I→I connections randomized</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>**Fig 3.** Recovery of damaged modular structure by spontaneous recalls. A Recovery experiment starting from randomized excitatory (A), inhibitory (B), or inhibitory plus excitatory-inhibitory (C) synaptic connections. In all the three cases, we study the partial recovery of the original modular organization of the synaptic weights over time. This recovery is mediated in all the cases by the emergence of spontaneous recalls, —transient events of partial synchronization between neurons associated with one of the two originally stored memories, highlighted in green for cluster 1 and orange for cluster 2.</figcaption></figure>

https://doi.org/10.1371/journal.pcbi.1012973.g003
suggests that memory items can be encoded in different types of connections with different effects on the intermediate or long-term storage and on the efficiency of recovering, which may echo the different types of memories present in the brain.

## Generalization of the model to more complex situations

So far, we have illustrated the results for networks trained with two stimuli, applied at separate neuronal sub-populations. We will now show that the model can be generalized. Firstly, the model is trained to an increasing number of memories in order to estimate its memory capacity. Secondly, the stability of the model is validated for larger network sizes and more stimuli, and lastly, we demonstrate the emergence of hub neurons by training the network with overlapping memory items.

**Memory capacity of the model** We begin by repeating the protocol in Fig 1D but now training the network to $M = 4$ non-overlapping stimuli, as shown in Fig 4A. We observe the same qualitative results as in the case with 2 stimuli, the only difference is that now four modules emerge in the connectivity matrix. Also in the present situation, the four neuronal assemblies display spontaneous recalls during the post-learning resting-state phase. A minor difference is that now the 20 inhibitory neurons split into 8 sub-populations, each made of 2–3 anti-Hebbian and Hebbian neurons. This opens the question of the memory capacity of the network, which seems limited by the number of inhibitory neurons.

In order to estimate the memory capacity of our model, we examine the ability of the network to store and recall an increasing number of memory items while varying the ratio of excitatory to inhibitory neurons. Specifically, connectivity matrices were initialized containing an arbitrary number $M$ of memory items (modules), from $M = 0$ to $M = 100$, and by varying the number of inhibitory neurons from $N_I = 0$ to $N_I = 100$. Since the size of the network is maintained constant to $N = 100$, the number of excitatory neurons is varied consequently as $N_E = 100 - N_I$. A network storing $M$ memory items is considered stable if (i) it displays asynchronous irregular firing and (ii) all the memory modules exhibit independent spontaneous recalls. A network with $M$ memories is considered unstable if at least one of the modules does not exhibit recalls and therefore disappears in the long-term.

The results are summarized in the stability diagram in Fig 4B. The red line separating the stable and the unstable regimes evidences that at least 2$M$ inhibitory neurons are needed to hold $M$ memories. This result implies that the network can learn and stabilize a maximum of $M^* = N/3 = 33$ memories (marked by a cyan star in Fig 4B). In particular, this maximal capacity corresponds to the case with $N_I = 66$ inhibitory neurons, 33 of them anti-Hebbian and 33 Hebbian, and $N_E = 33$ excitatory neurons. A simulation for this limit case is shown in S7 Text. Therefore, we conclude that in our model, the minimal condition for a memory to be robustly encoded and recalled is that the memory is associated to a triplet, made of at least one excitatory neuron, one Hebbian and one anti-Hebbian inhibitory neuron. For example, to store $M = 20$ memories (magenta dashed line in Fig 4B) the network needs to contain at least 20 excitatory neurons and 40 inhibitory neurons (20 Hebbian and 20 anti-Hebbian).

In sub-optimal scenarios, we find that for a given number of memory items $M < M^*$, different $E/I$ ratios can guarantee the long-term maintenance of the memories (connectivity patterns). All the possible $E/I$ ratios respecting these constraints can give rise to configurations in which the $M = 20$ memories are robustly stored. This amounts to have the excitatory to inhibitory ratio ranging from $N_E / N_I = 20 / 80 = 0.25$ up to $60 / 40 = 1.5$, where the total number of neurons is maintained equal to $N = 100$. For a case with $M = 10$ memories (green dashed line), the interval of acceptable ratios widens from $N_E / N_I = 10 / 90 = 0.111$
A Learning four stimuli

<figure><img src="image_16.png" /></figure>

B Stability diagram

<figure><img src="image_17.png" /></figure>

C Memory capacity

<table><thead><tr><th>Brain area</th><th>Species</th><th>Number of neurons N</th><th>E/I ratio</th><th>Hypothetical memory capacity</th></tr></thead><tbody><tr><td rowspan="2">Cortex</td><td>Human</td><td>16B</td><td>80/20</td><td>0.1N</td></tr><tr><td>Mouse</td><td>22M</td><td>80/20</td><td>0.1N</td></tr><tr><td rowspan="2">Cerebellum</td><td>Human</td><td>69B</td><td>80/20</td><td>0.1N</td></tr><tr><td>Mouse</td><td>42M</td><td>90/10</td><td>0.05N</td></tr><tr><td rowspan="2">Hippocampus</td><td>Human</td><td>48M</td><td>90/10</td><td>0.05N</td></tr><tr><td>Mouse</td><td>3M</td><td>95/5</td><td>0.025N</td></tr><tr><td rowspan="2">Amygdala</td><td>Human</td><td>13M</td><td>70/30</td><td>0.15N</td></tr><tr><td>Mouse</td><td>117K</td><td>60/40</td><td>0.2N</td></tr><tr><td rowspan="2">Thalamus</td><td>Human</td><td>20M</td><td>60/40</td><td>0.2N</td></tr><tr><td>Mouse</td><td>1M</td><td>80/20</td><td>0.1N</td></tr><tr><td rowspan="2">Basal ganglia</td><td>Human</td><td>415M</td><td>10/90</td><td>0.1N</td></tr><tr><td>Mouse</td><td>2M</td><td>20/80</td><td>0.2N</td></tr><tr><td rowspan="2">Striatum</td><td>Human</td><td>105M</td><td>5/95</td><td>0.05N</td></tr><tr><td>Mouse</td><td>1.7M</td><td>20/80</td><td>0.2N</td></tr><tr><td rowspan="2">Globus pallidus</td><td>Human</td><td>711K</td><td>10/90</td><td>0.1N</td></tr><tr><td>Mouse</td><td>250K</td><td>10/90</td><td>0.1N</td></tr></tbody></table>

**Fig 4.** Memory capacity of the model. (A) Training the network with *M* = 4 non-overlapping stimuli. The green, orange, pink and cyan brackets represent clusters 1, 2, 3 and 4. Results are qualitatively the same as for the example with two memory items. (B) Stability diagram of the model for a network of *N* = 100 neurons. The red line *M* = *N*<sub>l</sub>/2 separating the stability (orange) and instability (purple) regions,
represents an upper limit for the number of inhibitory neurons ($N_I = 2M$) needed to maintain $M$ independent memory items. The number of excitatory neurons in each realization is $N_E = 100 - N_I$ delimiting the non-accessible areas (white region) of the diagram. The star symbol marks the maximal memory capacity of the network, corresponding to $M^* = 33$ items with $N_I^* = 66$ inhibitory neurons (cyan lines, 33 anti-Hebbian and 33 Hebbian). Magenta and light-green lines highlight the ranges of number of inhibitory neurons (and therefore of the $N_I/N_E = N_I/(100 - N_I)$ ratios) that allow the stabilization of $M = 20$ and $M = 10$ memory items, respectively. (C) Table comparing the number of neurons and $E/I$ ratios of different parts of the human [62–74] and mouse [75–77] brain and their hypothetical memory capacity obtained by extrapolating the results of our model.

https://doi.org/10.1371/journal.pcbi.1012973.g004

to $80/20 = 4$. In summary, given that each memory requires a triplet of neurons, the memory capacity of our model is limited by the size of the smaller population of neurons, i.e. $\min[N_E, N_I^A, N_I^{HI}]$. For a network with the typical E to I ratio of 80/20, it implies that its memory capacity is controlled by the number of inhibitory neurons instead of the number of excitatory neurons.

The analysis of a few test cases reported in S4 Text for $M = 4$ memory items confirms the validity of the upper limit $M = 2N_I$. In particular, in S4 Text we studied the following situation: (i) each memory pattern is associated to one Hebbian and one anti-Hebbian inhibitory neuron, (ii) one memory pattern is missing an anti-Hebbian inhibitory neuron, and (iii) one memory item is missing one Hebbian inhibitory neuron.

The flexible relation between memory capacity and $E/I$ ratios found in our model invites to question why would nature favour the specific $E/I$ ratios observed in the mammalian brains, and whether these ratios might be related to their memory storage. To explore this question we computed the hypothetical memory capacities of the human and mouse brains, extrapolating the predictions from our model. Moreover, given that different parts of the brain are made of distinct number of neurons and $E/I$ ratios, we extended the calculations to distinguish the cortex, the cerebellum and a few subcortical regions. The results are shown in Fig 4C. The 80/20 excitatory to inhibitory ratio—which is mostly representative for the cerebral and cerebellar cortices—implies a hypothetical memory capacity of $M = 0.1N$, well below the theoretical maximal capacity of $M^* = 0.33N$. Still, given that the cortex and the cerebellum contain most of the neurons in the brain, in absolute terms this would imply that the human cortex and cerebellum have memory capacities of $M^{ctx} \approx 1.6 \times 10^9$ and $M^{cer} \approx 7.0 \times 10^9$ respectively, and $M^{ctx} \approx 2.2 \times 10^6$ and $M^{cer} \approx 10.5 \times 10^6$ in mice. Rather surprisingly, we find the lowest memory capacity in the Hippocampus, with $M = 0.050N$ and $M = 0.025N$ in humans and mice due to a reduced fraction of inhibitory neurons. The amygdala lies among the regions with largest memory capacity, $M = 0.15N$ and $M = 0.2N$ for humans and mice. This particularity of the amygdala could be related to the fact that this area is involved in the formation of emotional memories [78]. Nevertheless, the amygdala contains few neurons as compared to other regions, hence its absolute capacity would be definitely smaller.

**Robustness of the model for larger network sizes trained with multiple stimuli** The stability of a neuronal network can be compromised by the network size and the number of memories stored. Hence, to validate the robustness of our model we considered a network of 1000 neurons trained with 10 stimuli, Fig 5A. After training, the network was left to evolve spontaneously for a period of 4 hours. In Fig 5B, we observe the formation of 10 modules among the excitatory connections, each one associated with one group of Hebbian and one group of anti-Hebbian inhibitory neurons. The weight matrices remain practically unaltered over time up to 4 hours. This is additionally evidenced by the stability of the mean inter- and intra-cluster weights, see Fig 5C.
<figure><img src="image_18.png" /><figcaption>**Fig 5.** Stability of a network of N = 1000 neurons trained with M = 10 stimuli. (A) Post-learning raster plots of the neuronal activity at times: t=0h, t=2h and t=4h. In each plot, the spike count, normalized to the total number of neurons N and estimated over time bins of 0.2 sec, is displayed below the time axis. Selected spontaneous recalls are highlighted by colored shadows while a peak of global activity is highlighted by a grey shadow. Some of these events are zoomed on the right side of the panels. (B) Post-learning weights matrices at times: t=0h, t=2h and t=4h. (C) Post-learning evolution of mean intra- (solid lines) and inter- (dashed line) clusters weights for excitatory to excitatory (E-E dark red), excitatory to inhibitory (E-I red), anti-Hebbian inhibitory to excitatory (A-E dark blue), anti-Hebbian inhibitory to inhibitory (A-I cyan), Hebbian inhibitory to excitatory (H-E blue) and Hebbian inhibitory to inhibitory (H-I steel blue) connections.</figcaption></figure>

https://doi.org/10.1371/journal.pcbi.1012973.g005
The stability of the learned patterns depends on a few ingredients. On the one hand, the modules remain stable without merging or drifting [17]. This is possible because the anti-Hebbian inhibitory neurons ensure that recalls only occur one module at a time, depressing the other modules. If two modules were to undergo a recall simultaneously, their mutual synapses would become reinforced, causing their merging over repeated simultaneous events. Indeed, the spike counts remain below 5% of the network size, meaning that more memories are never fully recalled at the same time and that the network is far from being fully synchronized. This despite the occurrence of sporadic peaks of activity spread over the whole network, which however appear to be quite sparse (see the zoom of the grey shadowed interval in Fig 5A). A closer look at various recall events (see the enlargements of the coloured intervals in Fig 5A) shows that, usually, only the neurons of a given memory fire within a time window of 0.2 seconds.

On the other hand, for the modules to be maintained, a sufficient frequency of recalls is necessary. As just mentioned, only one module at a time can display a recall. Thus, the more memories are stored, the less frequently each module will be recalled. In consequence, the long-term maintenance of the modules is possible as long as the forgetting term *f* is sufficiently small, and inversely proportional to the number of modules *M*, (see S9 Text for a more detailed discussion). This experiment has also been repeated for different network sizes and number of stimuli, see S8 Text.

**Formation of hubs by training to overlapping memories** We finish the investigation of the model by considering the case in which two stimuli entrain an overlapping set of excitatory neurons and explore the possibility that these could encode for more than one stimulus [17,34,36–38]. This would correspond to a “mixed selectivity” scenario as it is often found in neurons of the prefrontal cortex [79,80]. During the training stage, populations P₁ and P₂ are now allowed to share eight excitatory neurons, see Fig 6A. Furthermore, stimulation to P₁ and P₂ is strictly alternated in order to facilitate the formation of the connections, instead of selecting randomly between P₁ and P₂, as done in the previous simulations.

The results in Fig 6B are similar to the ones before only that besides the formation of two clusters, now a set of structural hub neurons emerge in the connectivity matrix. These excitatory hubs are strongly connected to both modules. They are weakly affected by the feed-forward anti-Hebbian inhibition (since they belong to both populations), but are notably affected by the feedback Hebbian inhibition of the two populations. Fig 6C shows a schematic diagram of the initial and of the resulting connectivity.

Regarding the post-learning resting phase, see raster plot of Fig 6B, the network displays again a stable low-frequency asynchronous firing activity but with richer spatio-temporal patterns than in the previous examples. The spontaneous recalls are more varied now with events displaying: (i) synchronous spiking of one population including the hubs (event shaded in green), (ii) synchronous spiking of the population without the hubs (event shaded in orange), and (iii) synchronous spiking of the hub neurons alone (event shaded in pink). S5 Text shows similar results for a network trained with *M* = 4 overlapping stimuli.

In conclusion, the introduced model can be generalized to account for neurons that admit persistent mixed selectivity. This shows that, besides modular organization, the model can also incorporate centralized hierarchical organization which is a necessary ingredient for integration [1, 3, 8, 9]. In a previous study, based on phase oscillator models coupled via gap junctions and in presence of phase-difference-dependent plasticity (PDDP), the mixed selectivity was only a transient phenomenon. Indeed in Ref. [36], the PDDP tends to merge the hubs to one of the shared memories in the long-term, while in the spiking network with STDP here considered, the hubs are stable features of the connectivity structure.
<figure><img src="image_19.png" /><figcaption>**B Simulation**</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>**C Diagram**</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>**Fig 6.** Training the network with overlapping stimuli. (A) Experimental protocol for a network of N = 100 neurons trained with M = 2 stimuli that share 8 excitatory neurons. (B) Simulation and learning results. Connectivity matrices show the evolution of the synaptic weights leading to the emergence of two modules which overlap over 8 hub neurons. The raster plot shows the simulated neuronal activity of the network during the initial resting phase, the</figcaption></figure>
learning stage and the post-learning period. The activity during the post-learning phase is characterized by a variety of spontaneous recall events involving $P_1$ neurons and the hubs (green shadow), $P_2$ neurons without the hubs (orange shadow) and the hubs alone (pink shadow). (C) Schematic diagrams representing the formation of the synaptic connectivity with two stable populations of neurons and overlapping hubs. Each population is composed of at least a population of excitatory neurons $E_y$ and a population of hub neurons $H$ (in red), one population of Hebbian inhibitory neurons $I_x^H$ and one population of anti-Hebbian inhibitory neurons $I_x^A$ (in blue) ($x = 1, 2$). Dashed circles identify groups of neurons admitting synchronization events (memory recalls).

https://doi.org/10.1371/journal.pcbi.1012973.g006

## Summary and discussion

The architecture of brain's connectivity, the dynamics of neural populations and the learning capacities of neural networks are three fundamental topics of computational neuroscience. However, these topics are too often investigated separately one from another. In line with the research presented in [31–33,38], our aim here was to develop a learning model that exhibits the emergence of relevant architectural features but which remains *alive* in all phases. Meaning that neither the dynamical activity nor the plasticity evolution were frozen once the training is finalized. This is also referred as continuous learning. The model displays realistic firing patterns after the learning phase while the synaptic plasticity remains active—as it is the case in the brain *in-vivo*.

To achieve these objectives, we have introduced a network of excitatory and inhibitory QIF neurons with plastic synapses following simple STDP rules based only on local information, i.e. the pre- and post-synaptic spike times [45]. By targeting stimuli to distinct sub-populations—mimicking the segregated projections of different features into early sensory layers—the network developed a stable modular connectivity. Moreover, by allowing the stimuli to target also neurons in overlapping sub-populations, we observed the emergence of hub-like neurons displaying mixed selectivity. Furthermore, the learned connectivity patterns were robust against catastrophic forgetting over time despite plasticity remained active after the learning phase, and in the absence of any homeostatic or control mechanisms, in contrast to previous analyses [31–33,38]. As a matter of fact, we found that the key factor to guarantee the reinforcement and the long-term maintenance of the memories was the spontaneous occurrence of transient memory recalls in the post-learning neural dynamics. These events occurred at random times on top of an otherwise asynchronous and irregular background neuronal activity. In particular, the recalls acted as short, punctuated boosts to the synaptic adaptation which allowed the memory patterns to persist against natural forgetting.

## The role of feedback and feed-forward inhibition

Inhibition has often been considered to play a mere control function by avoiding pathological situations characterized by extremely high firing activity [81,82]. In our model, instead, inhibition also plays an active role in the formation and stabilization of the memories. Consequently, the memory capacity of the model not only depends on the number of excitatory neurons, but also on the number of inhibitory ones. Although inhibitory neurons account for only around 20% of neural cells in the human brain, GABAergic interneurons represent the majority of sub-classes present in the brain [83]. There is substantial evidence that inhibitory cells are regulated by different types of synaptic plasticity [84,85] promoting, for example, the creation of feed-forward [51,86] and feedback inhibitory circuits [52,86]. However, whether a direct relation exists between the type of plasticity and the various functionalities of inhibitory neurons is yet largely to be clarified [87]. Accordingly, we opted for exploring the behaviour of the model under Hebbian and anti-Hebbian inhibition.

During the training, the Hebbian inhibitory neurons formed internal feedback loops (with the excitatory neurons targeted by the same stimulus, Figs 1C and 1D) while the
anti-Hebbian neurons formed lateral feed-forward connections (across the populations that were targetted by different stimuli, Figs 1B and 1D). As a consequence, the Hebbian feed-back inhibition turned responsible for controlling the firing rate of the populations, which is crucial for the stabilization of the network activity into a realistic asynchronous irregular dynamics [40,88] and to prevent abnormal behaviour, e.g., the pathological high frequency firing observed in Fig 1B. Meanwhile, the anti-Hebbian feed-forward inhibition mediates the competition and selective activation across populations by, e.g., allowing the stimulated population (the stored memory item) to silence the activity of the other populations, representing other memory items.

These results resonate with recent empirical and modeling observations. Lagzi, et al. [42] showed via *in-vitro* experiments that parvalbumin-expressing (PV) and somatostatin-expressing (SOM) interneurons of the mouse frontal cortex follow symmetric and asymmetric Hebbian STDP respectively. Subsequent modeling suggested PV neurons to mediate homeostasis in excitatory activity, while SOM neurons build lateral inhibitory connections providing competition between excitatory assemblies. Guy et al. [87] performed *in-vivo* measurements of the responses of PV, SOM and vasoactive intestinal peptide-expressing (VIP) interneurons at the mouse barrel cortex and found differentiated angular selectivity functionality for each neuron type. In particular, whisker stimulation evoked direction-selective inhibition in a majority of SOM interneurons.

Previous models have compared the function of Hebbian and anti-Hebbian plasticities but these were investigated either separately [40,89–91] or assuming that one inhibitory neuron can be susceptible for both plasticity types [32]. Here, we considered two populations of inhibitory neurons—one population subject to Hebbian plasticity and the other to anti-Hebbian—and thus, we studied their joint impact on the adaptation and the dynamical behaviour of a neuronal network, allowing us to naturally associate both types of plasticity with given functions. In our model the feedback and feed-forward inhibitory circuits emerged spontaneously—through adaptation—and were not imposed a priori. In the light of our results, we conclude that the coexistence of Hebbian and anti-Hebbian inhibition is decisive for the formation of stable memory-related structural modules (assemblies) and for the onset of spontaneous memory recalls in the neuronal activity.

## The relevance of spontaneous recalls

The consolidation and long-term preservation of memories involve neurochemical changes that are modulated by the dynamical activity of the neurons and their neighbours [92–94]. Cortical and hippocampal activity during REM and non-REM sleep is characterized by low and high frequency firing events [95,96] that are relevant for the consolidation of daytime experiences [97–103]. The cortical activity during resting awake is typically characterised by asynchronous irregular dynamics, facilitating the integration of various sensory inputs and the formation of new associations between different information sources [104,105]. Additionally, short and random events of partial synchronous activation—occurring on top of the irregular firing background—are associated to spontaneous memory recalls (or retrieval) [39] and to the consolidation [40–42] of learned memories.

Once the learning phase has finalized, the dynamical behaviour of our model resembles that of awake resting-state *in-vivo* (Figs 2C-J). It shows both a persistent asynchronous irregular activity with low firing (≈ 0.0 – 5 Hz) and punctuated events of transient synchrony of higher rates (≈ 8 – 15 Hz). The STDP rules are effective models of the relation between the dynamical activity of neurons and the underlying biochemical processes leading to
synaptic plasticity [47]. STDP stresses that these biochemical changes—either for depres-
sion or potentiation—take effect primarily when neurons fire concurrently within short time
windows. In consequence, if the activity of a group of neurons is uncorrelated over time, the
memories they form tend to be slowly forgotten. In our model, this slow forgetting is com-
pensated by the occurrence of spontaneous (partial or complete) recalls of the previously
learned memories, happening at random times. These recalls act as short boosts reinforc-
ing the patterns of synaptic connectivity formed during the learning, and thus constitute
an autonomous mechanism for memory consolidation during rest. These results emphasize
the importance of considering dynamical models of learning which remain *alive* after train-
ing [31–33,38]—instead of freezing their activity and plasticity—in order to understand the
processes leading to memory consolidation and long-term maintenance.

It shall also be stressed that the coexistence of asynchronous irregular firing and memory recalls is an emergent dynamical property of our model, instead of being an enforced behaviour by the addition of internal mechanisms for frequency adaptation or by the application of external modulation. The asynchronous irregular activity emerges in our globally coupled model thanks to the three following: presence of non-identical neurons mimicking different neural properties, distributed synaptic weights, and external noise. However, analogous results can be obtained in randomly connected deterministic networks due to internal self-generated fluctuations as shown in S6 Text, as well as in previous analyses [31,32,38]. In a previous study with phase oscillators [36], the post-learning activity at rest showed a partially synchronous state similar to a limit cycle. Here the network remains stable in its asynchronous regime while the recalls spontaneously pop out at random times and order, allowing more numerous and complex information to be encoded and maintained. This behaviour naturally results from the interplay between the underlying structural organization—shaped by the learning process—and the background activity eliciting the transitions, somehow similar to the noise-driven state switching found in neural activity [106]. These results evidence the benefit of studying connectivity, dynamics and learning simultaneously, as they represent different faces of interdependent phenomena.

Memory capacity

Empirical and theoretical studies have associated inhibitory plasticity with the long-term stor-
age of memories [58–61]. From these indications one could deduce the idea that the (long-
term) memory capacity of the brain might be related to the quantity of inhibitory neurons.
We have shown that the memory capacity of our model depends on the number of both exci-
tatory and inhibitory neurons. The maximal storage capacity of our model is M* = 0.333N
memory items (Fig 4B), where N is the total number of neurons. This represents more than
twice the memory capacity of the classical Hopfield model (≈ 0.14N [107,108]) which does
not respect the Dale's principle—it allows the connections to take either positive or nega-
tive values, thus ignoring the separation of neurons into excitatory and inhibitory classes. It
should be stressed that the value M* should be considered as a theoretical limit. It arises from
the fact that, in our model, each memory needs to be associated to at least three neurons: a
triplet formed by one excitatory, one Hebbian inhibitory and one anti-Hebbian inhibitory
neuron. While in correspondence of this limit values the stability of all the memories is guar-
anteed, achieving this capacity might be rather implausible for biological neural networks.
For example, a real brain operating at this limit would be very fragile since the lesion of a sin-
gle neuron would be sufficient to erase a memory. Also, this limit requires the E/I ratio to
be 33/66, while in the human cortex this is known to be 80/20. For this ratio, the maximal
capacity of our model is indeed 0.1N.
An overview of the known variability of *E/I* ratios across different parts of the human brain (Table in Fig 4C) reveals that smaller regions such as the amygdala, thalamus, striatum and globus pallidus tend to have a higher proportion of inhibitory neurons than the cortex or the cerebellum. Extrapolating the results for the memory capacity from our model, it allows us to speculate that regions with fewer resources for memory storage (in terms of total number of neurons), may compensate by allowing for a more favourable *E/I* ratio. Similarly, the mouse admits a more favourable ratio than humans in some areas such as the basal ganglia, striatum, amygdala or the hippocampus. While this interpretation is at the moment largely hypothetical, it may reflect a biological strategy for resource optimisation. Furthermore, it shall be noted that memory capacity is further affected by the complexity of the memories; with simple ones requiring less neurons to be stored and complex memories each requiring more neurons. Hence, the empirically observed *E/I* ratios may also reflect a compromise between the number of memories and the complexity of the information that needs to be stored.

Real neurons generally respond to—and encode for—multiple inputs and memories [109]. This mixed selectivity plays a crucial role in complex cognitive tasks allowing the brain to simultaneously represent and integrate multiple sources of information [79,110]. In addition to retaining segregated memories, our model also exhibits the possibility of learning and recalling complex memories admitting mixed selectivity, see Fig 6. The neurons supporting mixed selectivity can be considered as hub neurons [111] connecting between the stored clusters and facilitating the ability to transmit and integrate information [1,112]. The presence of hub neurons does not increase the memory capacity of a network but instead, they allow for more complex information to be encoded by establishing associations between memories.

## Limitations and outlook

Despite the model was developed to respect several biological constraints and to satisfy some realistic behavioural aspects, other choices and limitations could be implemented in future refinements. We omitted possible evolutionary aspects and assumed that only synaptic plasticity to be responsible for shaping the structure of the connectivity. Sensory systems organize information spatially with neighbouring neurons representing similar response properties and forming, e.g., retinotopic, tonotopic or somatotopic maps [113–115]. However, in the present model the neurons form a graph with no spatial embedding. It would be thus a natural step to extend the model in the future with spatially distributed neurons, following biologically representative spatial constraints.

The long-range white matter connections between distant cortical regions are mainly formed by axonal bundles of excitatory pyramidal neurons [116,117], although some GABAergic cells have also been found to project across different brain regions [111,118]. When our model was trained with stimuli targeting separate neuronal populations, excitatory neurons formed local connections and only the anti-Hebbian inhibitory neurons developed “long-range” projections to other clusters. While this architecture is consistent with the lateral inhibition found in circuits for selectivity and decision-making [87,119,120], it is not representative of the excitatory long-range white matter fibers in the cortex. However, in the model, cross-modular excitatory connections appeared when the stimuli acted on overlapping populations, leading to the formation of excitatory hubs. While the reasons for the organization of excitatory and inhibitory cross-modular connectivities requires further investigation, extensions of the model here presented might be relevant to explore those mechanisms. In fact, both the organization of micro-/macroscopic connectivities and the development of
short-/long-term memories, might be governed by distinct rules which could be separately implemented in the model.

## Methods

This section describes the spiking neuronal network model governing the dynamics of the neurons and the learning rules used for the adaptation of synaptic weights, as well as the microscopic and macroscopic indicators employed to characterize the network states and dynamics in the paper.

### Spiking neuronal network model

We consider a network of QIF neurons, pulse coupled via exponentially decaying post-synaptic potentials and in the presence of STDP. Unless otherwise specified, the network will be composed of 80% (20%) excitatory (inhibitory) neurons as usually observed in the human cortex [121]. Depending on the plasticity rules controlling the synaptic strengths of the connections, three neural sub-populations can be identified depending on the nature of the pre-synaptic neurons:

* excitatory neurons subject to asymmetric Hebbian STDP;

* inhibitory neurons subject to symmetric Hebbian STDP;

* inhibitory neurons subject to symmetric anti-Hebbian STDP.

The evolution of the membrane potential $V_i$ of each neuron ($i = 1, ..., N$) is described by the following ordinary differential equation:

$$ \tau_m \dot{V}_i = V_i^2(t) + \eta_i + g_c S_i^e(t) + g_{hi} S_i^{hi}(t) + g_{ai} S_i^{ai}(t) + I_i(t) + \xi_i(t), \quad (1) $$

where $\tau_m = \tau_0$ ms (with $\tau_0 = 20$) is the membrane time constant; $\eta_i \sim N(0.0, (\pi\tau_0)^2)$ are the neuronal excitabilities, chosen to have an average neuronal firing rate at rest of around 1 Hz, $I_i(t) = \{0, (50\pi\tau_0)^2\}$ are the external DC currents, leading the neurons to fire around 50 Hz whenever stimulated and $\xi_i(t) \sim N(0.0, (4\pi\tau_0)^2)$ is a Gaussian white noise term tuned to induce a firing variability of $\approx 4$ Hz. Whenever the membrane potential $V_i$ reaches infinity, a spike is emitted and $V_i$ is reset to $-\infty$. In the absence of synaptic coupling, external DC current and noise, the QIF model displays excitable dynamics for $\eta_i < 0$, while for positive $\eta_i$, it behaves as an oscillator with period $T_i^{(0)}/\tau_m = \pi/\sqrt{\eta_i}$ [54].

The synaptic dynamics is mediated by the global synaptic strengths: $g_e = 100$, $g_{hi} = 400$ and $g_{ai} = 200$ for the excitatory, the Hebbian and anti-Hebbian inhibitory neurons, respectively. Finally, the evolution of the excitatory, Hebbian inhibitory and anti-Hebbian inhibitory synaptic currents $S_i^e(t)$, $S_i^{hi}(t)$ and $S_i^{ai}(t)$ is given by

$$ \tau_d^e \dot{S}_i^e = -S_i^e + \frac{\tau_d^e}{N_e} \sum_{j=1}^{N_e} \sum_{n} w_{ij}(t) \delta(t - t_j^{(n)}), \quad (2) $$

$$ \tau_d^i \dot{S}_i^{hi} = -S_i^{hi} + \frac{\tau_d^i}{N_{hi}} \sum_{j=1}^{N_{hi}} \sum_{n} w_{ij}(t) \delta(t - t_j^{(n)}), \quad (3) $$

$$ \tau_d^i \dot{S}_i^{ai} = -S_i^{ai} + \frac{\tau_d^i}{N_{ai}} \sum_{j=1}^{N_{ai}} \sum_{n} w_{ij}(t) \delta(t - t_j^{(n)}), \quad (4) $$
where $\tau_d^e = 2$ ms ($\tau_d^i = 5$ ms) are the exponential time decay for the excitatory (inhibitory) post-synaptic potentials; $w_{ij}(t)$ are the plastic coupling weights from neuron j towards neuron i, whose dynamics is described in the following and $t_j^{(n)}$ is the n-th spike time of the j-th pre-synaptic neuron, and $\delta(t)$ is the Dirac delta function.

We consider a network of N all-to-all connected neurons without self-connections and $N = N_e + N_{hi} + N_{ai} = 100$, where $N_e = 80$, $N_{hi} = 10$ and $N_{ai} = 10$ are the number of excitatory, Hebbian inhibitory and anti-Hebbian inhibitory neurons, respectively.

We integrate Eqs. (1), (2), (3) and (4) via a stochastic Euler scheme [122] with time step $dt$ (see Table 1 for its value). Whenever $V_i(t)$ overcomes a threshold value $V_p = 10$, we approximate the n-th firing time of neuron i as $t_i^{(n)} = t + \frac{1}{V_i} \tau_m$, where $\frac{1}{V_i} \tau_m$ corresponds to the time needed to reach $+\infty$ from the value $V_p$. Furthermore, the neuron is reset to $V_r = -10$ and held to such value for a time interval $\frac{2}{V_i} \tau_m$ seconds corresponding to the time needed for the neuron to reach $V_i(t) = \infty$ from $V_p$ and to return to $V_r$ after being reset to $-\infty$ [82].

The choice of parameters sets the neurons into a low firing state with a relatively large variability ranging from 0.0 to 8 Hz, analogous to those observed in the cortex at rest. The external drives $\{I_i(t)\}$ turn the neuronal activity into high frequency oscillations in the $\gamma$ range of approximately 50 Hz, typical of high-order perceptual activity [123,124].

## STDP rules

The time evolution of the synaptic weights $w_{ij}(t)$ is controlled by STDP rules which depend on the time difference $\Delta t = t_i - t_j$ between the last spikes of the post-synaptic neuron i and the pre-synaptic neuron j [45,47–49].

The potentiation of the synapses is controlled by the plasticity function

$$ \Lambda^+(\Delta t) = \begin{cases} \Lambda(\Delta t), & \text{if } \Lambda(\Delta t) \ge 0, \\ 0, & \text{if } \Lambda(\Delta t) < 0, \end{cases} \qquad (5) $$

while their depression is controlled by

$$ \Lambda^-(\Delta t) = \begin{cases} 0, & \text{if } \Lambda(\Delta t) \ge 0, \\ \Lambda(\Delta t), & \text{if } \Lambda(\Delta t) < 0, \end{cases} \qquad (6) $$

where $\Lambda(\Delta t)$ entering in Eqs. (5) and (6) depends on the nature of the pre-synaptic neuron.

For the pre-synaptic excitatory neurons we use an asymmetric Hebbian STDP function $\Lambda_e(\Delta t)$ [45]. This function affects the evolution of the weights in a causal way: when the pre-synaptic neuron j emits a spike before (after) the post-synaptic neuron i, the synaptic weight $w_{ij}$ increases (decreases) [125]. The function $\Lambda_e(\Delta t)$ is depicted in Fig 7A, and its explicit expression reads as

$$ \Lambda_e(\Delta t) = \begin{cases} A_+ e^{-\frac{\Delta t}{\tau_+}} - A_- e^{-\frac{4\Delta t}{\tau_+}} - f, & \text{for } \Delta t \ge 0, \\ A_+ e^{\frac{4\Delta t}{\tau_-}} - A_- e^{\frac{\Delta t}{\tau_-}} - f, & \text{for } \Delta t < 0, \end{cases} \qquad (7) $$

with the time constants $\tau_+ = 0.02$ sec and $\tau_- = 0.05$ sec, the amplitudes $A_+ = 5.296$ and $A_- = 2.949$.

The term f models a natural slow forgetting of the memories [126,127] by causing a small but constant depression of the weights. We assume that f is inversely proportional to the number M of encoded items: namely, $f = f_0/M$, with $f_0 = 0.2$. This in order to allow all the stored memory items to be randomly recalled during the post-learning resting phase (for
more details, see S9 Text). Apart from the term f, all the other parameters are not modified in the simulations.

For pre-synaptic Hebbian inhibitory neurons, we use a symmetric Hebbian STDP function $\Lambda_{hi}(\Delta t)$ [40,51,86,90,91,128], which takes the form of a Ricker wavelet function (or Mexican hat), potentiating (depressing) weights of neurons spiking in a correlated (uncorrelated) way. $\Lambda_{hi}(\Delta t)$ is shown in Fig 7B and it reads as

$$ \Lambda_{hi}(\Delta t) = A \left[ 1 - \left( \frac{\Delta t}{\tau} \right)^2 \right] e^{-\frac{\Delta t^2}{2\tau^2}} - f, \quad (8) $$

with the time constant $\tau = 0.1$ sec, the amplitude $A = 3$ and the forgetting term $f = 0.1$.

For pre-synaptic anti-Hebbian inhibitory neurons we use a symmetric anti-Hebbian STDP function $\Lambda_{ai}(\Delta t)$ [52,85,86,89,91,129] which corresponds to a reverse Ricker wavelet function (or reverse Mexican hat), potentiating (depressing) weights of neurons spiking in a uncorrelated (correlated) way. The function $\Lambda_{ai}(\Delta t)$ is shown in Fig 7C and it takes the following expression:

$$ \Lambda_{ai}(\Delta t) = -A \left[ 1 - \left( \frac{\Delta t}{\tau} \right)^2 \right] e^{-\frac{\Delta t^2}{2\tau^2}} + f, \quad (9) $$

with the time constant $\tau = 0.1$ sec and the amplitude $A = 3$. In this case, the forgetting term $f = 0.1$ allows to have a constant small potentiation (the rule being anti-Hebbian) of the weights whatever the spike timing difference.

<figure><img src="image_22.png" /><figcaption>**Fig 7. Plasticity and soft bound functions.** Plasticity functions: (A) Hebbian asymmetric STDP $\Lambda_c(\Delta t)$ (Eq. (7)); (B) Hebbian symmetric STDP $\Lambda_{hi}(\Delta t)$ (Eq. (8)); (C) anti-Hebbian symmetric STDP $\Lambda_{ai}(\Delta t)$ (Eq. (9)). Soft bound functions for excitatory (D) and inhibitory (E) neurons. The bounds for potentiation are shown in red and for depression in blue. The displayed functions refer to $M = 2$ encoded stimuli, i.e. to $f = 0.1$.</figcaption></figure>

https://doi.org/10.1371/journal.pcbi.1012973.g007
## Adaptation of the synaptic weights

In this sub-section we explain in details the evolution of the synaptic weights. In particular, when the pre-synaptic neuron j or the post-synaptic neuron i spikes at time t, the weight $w_{ij}$ is updated according to the following equations:

$$w_{ij}(t^+) = w_{ij}(t^-) + \gamma_l \Delta w_{ij}, \quad (10)$$

with

$$\Delta w_{ij} = (-1)^{a_q} [\tanh(\lambda(w_q^l - w_{ij})) * \Lambda_q^{x(q)}(\Delta t) + \tanh(\lambda(w_{ij} + w_q^u)) * \Lambda_q^{y(q)}(\Delta t)], \quad (11)$$

where q denotes if the pre-synaptic neuron is excitatory $q = e$ or Hebbian (anti-Hebbian) inhibitory $q = hi$ ($q = ai$). For excitatory (inhibitory) neurons we set $w_q^l = 1$ ($w_q^l = 0$) and $w_q^u = 0$ ($w_q^u = 1$), thus ensuring that the excitatory (inhibitory) couplings are defined within the interval $w_{ij} \in [0:1]$ ($w_{ij} \in [-1:0]$). Moreover, $a_e = 2$ and $a_{hi} = a_{ai} = 1$ while $x(e) = y(hi) = y(ai) = +$ and $y(e) = x(hi) = x(ai) = -$, thus, for inhibitory synapses, the plasticity functions $\Lambda^+(\Delta t)$ and $\Lambda^-(\Delta t)$ are exchanged and multiplied by $-1$ since potentiation (depression) of inhibitory weights makes them converge towards $-1$ (0). Furthermore, $\gamma_l = 0.005$ is the dimensionless learning rate for the adaptation, while the parameter $\lambda = 100$ controls the slope of the soft bound function $\tanh(\lambda x)$.

These non-linear functions, depicted in Figs 7D and 7E, model the saturation of the synaptic weights $|w_{ij}|$ between 0 and 1 [130]. The saturation is obtained by allowing for synaptic depression (potentiation) dominating the potentiation (depression) term for large (small) values of the weights [47,131]. The steep slope of these functions (controlled by the parameter $\lambda$) guarantees a dynamical evolution of the synaptic weights even for large values of $w_{ij}$ [132]. In this sense, the functions here used are at the limit between a soft and a hard bound.

We note that the firing activity of the neurons directly impacts the adaption rate and that the synaptic weights are always subject to adaptation, unlike in conventional artificial neural networks. This differs from the previous study with phase oscillators and PDDP [36], where two different learning timescales were needed to allow for the storing and maintenance of the memory items.

All network parameters employed throughout this study are summarized in Table 1.

## Microscopic and macroscopic indicators

In this sub-section, we define the indicators employed to characterize the network dynamics at a microscopic and macroscopic level.

**Microscopic Indicators** The behaviour of single neurons was quantified by their instantaneous firing rates $\nu_j(t)$ defined as

$$\nu_j(t) = \frac{n_j^{sp}(t)}{T}. \quad (12)$$

Here, $n_j^{sp}(t)$ is the number of spikes emitted by neuron j (i.e. the spike count) in the time interval $[t: t+T]$, with $T = 0.05$ sec.

For a population of $N_p$ neurons, their instantaneous activity can be characterized as follows

$$\nu_p(t) = \frac{1}{N_p} \sum_{j=1}^{N_p} \nu_j(t). \quad (13)$$
<table><caption>Table 1. Parameters for the network of QIF neurons.</caption><thead><tr><th>Parameters</th><th>Values</th></tr></thead><tbody><tr><td>N</td><td>100</td></tr><tr><td>N<sub>e</sub></td><td>80</td></tr><tr><td>N<sub>hi</sub></td><td>10</td></tr><tr><td>N<sub>ai</sub></td><td>10</td></tr><tr><td>g<sub>e</sub></td><td>100</td></tr><tr><td>g<sub>hi</sub></td><td>200</td></tr><tr><td>g<sub>ai</sub></td><td>400</td></tr><tr><td>η</td><td>N(0.0, (πτ<sub>0</sub>)<sup>2</sup>)</td></tr><tr><td>I(t)</td><td>{0, (50πτ<sub>0</sub>)<sup>2</sup>}</td></tr><tr><td>ξ(t)</td><td>N(0.0, (4πτ<sub>0</sub>)<sup>2</sup>)</td></tr><tr><td>V<sub>p</sub></td><td>10</td></tr><tr><td>V<sub>r</sub></td><td>-10</td></tr><tr><td>τ<sub>0</sub></td><td>0.02</td></tr><tr><td>τ<sub>m</sub></td><td>0.02 sec</td></tr><tr><td>τ<sub>d</sub><sup>e</sup></td><td>0.002 sec</td></tr><tr><td>τ<sub>d</sub><sup>l</sup></td><td>0.005 sec</td></tr><tr><td>τ<sub>+</sub></td><td>0.02 sec</td></tr><tr><td>τ<sub>-</sub></td><td>0.05 sec</td></tr><tr><td>τ</td><td>0.1 sec</td></tr><tr><td>dt</td><td>0.001 sec</td></tr><tr><td>γ<sub>l</sub></td><td>0.005</td></tr><tr><td>A<sub>+</sub></td><td>5.296</td></tr><tr><td>A<sub>-</sub></td><td>2.949</td></tr><tr><td>A</td><td>3</td></tr><tr><td>f</td><td>0.1</td></tr><tr><td>λ</td><td>100</td></tr></tbody></table>

https://doi.org/10.1371/journal.pcbi.1012973.t001

The firing rate variability has been measured in terms of the coefficient of variation $CV_j = \frac{\sigma_j}{\mu_j}$ where $\mu_j$ is the mean interspike interval (ISIs) of the neuron j and $\sigma_j$ its standard deviation. For a perfectly periodic firing $CV_j = 0$, while for a Poissonian process $CV_j = 1$.

**Macroscopic indicators** The degree of synchronization in the network was quantified by the complex Kuramoto order parameter [133]

$$Z(t) = R(t)e^{i\Phi(t)} = \frac{1}{N} \sum_{j=1}^{N} e^{i\theta_j(t)}, \quad (14)$$

where $R(t)$ ($\Phi(t)$) represents the modulus (phase) of the macroscopic indicator. The modulus $R$ is employed to characterize the level of phase synchronization in the network: $R>0$ ($R = 1$) for a partially (fully) synchronized network, while $R \approx O(1/\sqrt{N})$ for an asynchronous dynamics due to finite size effects.

To associate a continuous phase $\theta_j(t) \in [0 : 2\pi]$ to the spiking activity of neuron j, we proceed in the following way:

$$\theta_j(t) = 2\pi \frac{(t - t_j^{(n)})}{(t_j^{(n+1)} - t_j^{(n)})} \qquad t_j^{(n)} \le t \le t_j^{(n+1)}, \quad (15)$$

with $t_j^{(n)}$ the n-th firing time of neuron j.
The mean rate of variation of the synaptic weights was estimated as

$$K(t) = \frac{1}{N * (N - 1)} \sum_{i=1}^{N} \sum_{j \neq i}^{N} \frac{[w_{ij}(t + \Delta t) - w_{ij}(t)]}{\Delta t}, \quad (16)$$

where $w_{ij}(t)$ and $w_{ij}(t + \Delta t)$ are the synaptic coupling weights from neuron j to i at times t and $t + \Delta t$ respectively, $\Delta t = 0.1$ sec. The normalization term is $N*(N-1)$ since we consider an all-to-all connected network without autapses. The parameter $K(t)$ takes positive (negative) values for an overall increase (decrease) in the weight connectivity.

## Supporting information

**S1 Text.** Learning of 2 stimuli with an untrained population.
(PDF)

**S2 Text.** Learning of 2 stimuli with randomly stimulated neurons.
(PDF)

**S3 Text.** Learning of 2 stimuli with random amplitude frequencies.
(PDF)

**S4 Text.** Evolution of a network initially made of four structural modules in absence of any stimulation.
(PDF)

**S5 Text.** Learning of 4 overlapping stimuli.
(PDF)

**S6 Text.** A simulation for a network composed of $N = 20000$ neurons and a simulation for a network composed of $N = 1000$ neurons and sparse connections.
(PDF)

**S7 Text.** Learning of 33 stimuli in a network composed of $N = 100$ neurons.
(PDF)

**S8 Text.** Post-learning network connections stability for networks of size $N = 2000$, $N = 1000$, $N = 500$ and $N = 200$ neurons, trained to $M = 20$, $M = 10$, $M = 5$ and $M = 2$ memories.
(PDF)

**S9 Text.** Estimation of the time needed on average to forget the reinforcement of the synaptic weights induced by a single recall event.
(PDF)

## Acknowledgments

The authors thank Matthieu Gilson for useful discussions.

## Author contributions

**Conceptualization:** Raphaël Bergoin, Alessandro Torcini, Gorka Zamora-López.

**Formal analysis:** Raphaël Bergoin.

**Funding acquisition:** Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López.
**Investigation:** Raphaël Bergoin.

**Methodology:** Raphaël Bergoin, Alessandro Torcini, Gorka Zamora-López.

**Project administration:** Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López.

**Software:** Raphaël Bergoin.

**Supervision:** Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López.

**Validation:** Raphaël Bergoin, Alessandro Torcini, Gorka Zamora-López.

**Visualization:** Raphaël Bergoin.

**Writing – original draft:** Raphaël Bergoin.

**Writing – review & editing:** Raphaël Bergoin, Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López.

## References

1. Zamora-López G, Zhou C, Kurths J. Cortical hubs form a module for multisensory integration on top of the hierarchy of cortical networks. Front Neuroinform. 2010;4:1. doi: 10.3389/neuro.11.001.2010 PMID: 20428515

2. Varshney LA, Chen BL, Paniagua E, Hall DH, Chklovskii DB. Structural properties of the *Caenorhabditis elegans* neuronal network. PLoS Comput Biol. 2011;7(2):e1001066. doi: 10.1371/journal.pcbi.1001066. PMID: 21304930

3. Sporns O. Network attributes for segregation and integration in the human brain. Curr Opin Neurobiol. 2013;23(2):162–71. doi: 10.1016/j.conb.2012.11.015 PMID: 23294553

4. Hilgetag CC, Goulas A. 'Hierarchy' in the organization of brain networks. Philos Trans R Soc Lond B Biol Sci. 2020;375(1796):20190319. doi: 10.1098/rstb.2019.0319 PMID: 32089116

5. Suarez LE, Yovel Y, van den Heuvel M, Sporns O, Assaf Y, Lajoie G, et al. A connectomics-based taxonomy of mammals. Elife. 2022;11:e78635. doi: 10.7554/eLife.78635 PMID: 36342363

6. Lin A, Yang R, Dorkenwald S, Matsliah A, Sterling AR, Schlegel P, et al. Network statistics of the whole-brain connectome of Drosophila. Nature. 2024;634:153–65. doi:https://doi.org/10.1038/s41586-024-07968-y PMID: 39358527

7. Sporns O, Tononi GM. Classes of network connectivity and dynamics. Complexity. 2001;7(1):28–38. https://doi.org/10.1002/cplx.10015

8. Shanahan M. Broadcast and the network. In: Embodiment and the inner life: Cognition and consciousness in the space of possible minds. Oxford University Press; 2010.

9. Zamora-López G, Zhou C, Kurths J. Exploring brain function from anatomical connectivity. Front Neurosci. 2011;5:83. doi: 10.3389/fnins.2011.00083 PMID: 21734863

10. Scannell JW, Blakemore C, Young MP. Analysis of connectivity in the cat cerebral cortex. J Neurosci. 1995;15(2):1463–83. doi: 10.1523/JNEUROSCI.15-02-01463.1995 PMID: 7869111

11. Hilgetag CC, Burns GA, O'Neill MA, Scannell JW, Young MP. Anatomical connectivity defines the organization of clusters of cortical areas in the macaque and the cat. Philos Trans R Soc Lond B Biol Sci. 2000;355(1393):91–110. doi: 10.1098/rstb.2000.0551 PMID: 10703046

12. Meunier D, Lambiotte R, Bullmore ET. Modular and hierarchically modular organization of brain networks. Front Neurosci. 2010;4:200. doi: 10.3389/fnins.2010.00200 PMID: 21151783

13. Senden M, Deco G, De Reus MA, Goebel R, Van Den Heuvel MP. Rich club organization supports a diverse set of functional network configurations. Neuroimage. 2014;96:174–82. doi: 10.1016/j.neuroimage.2014.03.066 PMID: 24699017

14. Bertolero MA, Yeo BTT, D'Esposito M. The modular and integrative functional architecture of the human brain. Proc Natl Acad Sci U S A. 2015;112(49):E6798–807. doi: 10.1073/pnas.1510619112 PMID: 26598686

15. Rubinov M, Sporns O, van Leeuwen C, Breakspear M. Symbiotic relationship between brain structure and dynamics. BMC Neurosci. 2009;10:55. doi: 10.1186/1471-2202-10-55 PMID: 19486538
16. Damicelli F, Hilgetag CC, Hütt MT, Messé A. Topological reinforcement as a principle of modularity emergence in brain networks. Network Neuroscience. 2019;3(2):589–605. doi: 10.1162/netn_a_00085 PMID: 31157311
17. Manz P, Memmesheimer RM. Purely STDP-based assembly dynamics: Stability, learning, overlaps, drift and aging. PLoS Comput Biol. 2023;19(4):e1011006. doi: 10.1371/journal.pcbi.1011006 PMID: 37043481

18. Li J, Bauer R, Rentzepis I, van Leeuwen C. Adaptive rewiring: a general principle for neural network development. Front Netw Physiol. 2024;4:1410092. doi: 10.3389/fnetp.2024.1410092 PMID: 39534101

19. Kirkby LA, Sack GS, Firl A, Feller MB. A role for correlated spontaneous activity in the assembly of neural circuits. Neuron. 2013;80(5):1129-44. doi: 10.1016/j.neuron.2013.10.030 PMID: 24314725

20. Shanahan M. Metastable chimera states in community-structured oscillator networks. Chaos. 2010;20:013108. doi: 10.1063/1.3305451. PMID: 20370263

21. Shanahan M. A spiking neuron model of cortical broadcast and competition. Conscious Cogn. 2007;17:288-303. doi: 10.1016/j.concog.2006.12.005 PMID: 17317220

22. Schaub MT, Bileh YN, Anastassiou CA, Koch C, Barahona M. Emergence of Slow-Switching Assemblies in Structured Neuronal Networks. PLoS Comput Biol. 2015;11(7):e1004196. doi: 10.1371/journal.pcbi.1004196 PMID: 26176664

23. Fuster J. Unit activity in prefrontal cortex during delayed-response performance: neural correlates of transient memory. J Neurophysiol. 1973;36:61. doi: 10.1152/jn.1973.36.1.61 PMID: 4196203

24. Miyashita Y, Hayashi T. Neural representation of visual objects: encoding and top-down activation. Curr Opin Neurobiol. 2000;10:187-94. doi: 10.1016/S0959-4388(00)00071-4 PMID: 10753793

25. Hubel DH, Wiesel TN. Receptive fields, binocular interactions and functional architecture in the cat's visual cortex. J Physiol. 1962;160:106-54. doi: 10.1113/jphysiol.1962.sp006837 PMID: 14449617

26. Del Giudice P, Mattia M. Long and short-term synaptic plasticity and the formation of working memory: a case study. Neurocomputing. 2001;38:1175-80. https://doi.org/10.1016/S0925-2312(01)00557-4

27. Del Giudice P, Fusi S, Mattia M. Modelling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses. J Physiol Paris. 2003;97(4-6):659-81. doi: 10.1016/j.jphysparis.2004.01.021 PMID: 15242673

28. Amit DJ, Mongillo G. Spike-driven synaptic dynamics generating working memory states. Neural Comput. 2003;15(3):565-96. doi: 10.1162/089976603321192086 PMID: 12620158.

29. Morrison A, Aertsen A, Diesmann M. Spike-timing-dependent plasticity in balanced random networks. Neural Comput. 2007;19(6):1437-1467. doi: 10.1162/neco.2007.19.6.1437 PMID: 17444756

30. Clopath C, Büsing L, Vasilaki E, Gerstner W. Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nat Neurosci. 2010;13(3):344-352. doi: 10.1038/nn.2479 PMID: 20098420

31. Litwin-Kumar A, Doiron B. Formation and maintenance of neuronal assemblies through synaptic plasticity. Nat Commun. 2014;5:5319. doi: 10.1038/ncomms6319. PMID: 25395015

32. Zenke F, Agnes EJ, Gerstner W. Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nat Commun. 2015;6:6922. doi: 10.1038/ncomms7922 PMID: 25897632

33. Ocker GK, Doiron B. Training and spontaneous reinforcement of neuronal assemblies by spike timing plasticity. Cerebral Cortex. 2019;29(3):937-51. doi: 10.1093/cercor/bhy001 PMID: 29415191

34. Fauth MJ, van Rossum MCW. Self-organized reactivation maintains and reinforces memories despite synaptic turnover. eLIFE. 2019;8:e43717. doi: 10.7554/eLife.43717 PMID: 31074745

35. Miehl C, Onasch S, Festa D, Gjorgijeva J. Formation and computational implications of assemblies in neural circuits. J Physiol. 2023;601(15):3071-90. doi: 10.1113/JP282750 PMID: 36068723

36. Bergoin R, Torcini A, Deco G, Quoy M, Zamora-López G. Inhibitory neurons control the consolidation of neural assemblies via adaptation to selective stimuli. Sci Rep. 2023;13(1):6949. doi: 10.1038/s41598-023-34165-0 PMID: 37117236

37. Bergoin R. The role of inhibitory plasticity in the formation and the long-term maintenance of neural assemblies and memories. CY Cergy Paris Université; Universitat Pompeu Fabra; 2023.

38. Yang X, La Camera G. Co-existence of synaptic plasticity and metastable dynamics in a spiking model of cortical circuits. PLoS Comput Biol. 2024;20(7):e1012220. doi: 10.1371/journal.pcbi.1012220 PMID: 38106233
39. Gu Y, Gong P. The dynamics of memory retrieval in hierarchical networks. J Comput Neurosci. 2016;40(3):247-68. doi: 10.1007/s10827-016-0595-7 PMID: 26922679

40. Vogels TP, Sprekeler H, Zenke F, Clopath C, Gerstner W. Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science. 2011;334(6062):1569-73. doi: 10.1126/science.1211095 PMID: 22075724

41. Carrillo-Reid L, Yang W, Bando Y, Peterka DS, Yuste R. Imprinting and recalling cortical ensembles. Science. 2016;353(6300):691-4. doi: 10.1126/science.aaf7560 PMID: 27516599

42. Lagzi F, Bustos MC, Oswald AM, Doiron B. Assembly formation is stabilized by Parvalbumin neurons and accelerated by Somatostatin neurons. BioRxiv. preprint. 2021; pp. 2021-09. doi: 10.1101/2021.09.06.459211

43. Van Vreeswijk C, Sompolinsky H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science. 1996;274(5293):1724-6. doi: 10.1126/science.274.5293.1724 PMID: 8939866.

44. Brunel N. Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J Comput Neurosci. 2000;8:183-208. doi: 10.1023/a:1008925309027 PMID: 10809012

45. Bi G, Poo M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci. 1998;18(24):10464-72. doi: 10.1523/JNEUROSCI.18-24-10464.1998 PMID: 9852584

46. Bi Gq, Poo Mm. Synaptic modification by correlated activity: Hebb's postulate revisited. Annu Rev Neurosci. 2001;24(1):139-66. doi: 10.1146/annurev.neuro.24.1.139 PMID: 11283308

47. Sjöström J, Gerstner W. Spike-timing dependent plasticity. Scholarpedia. 2010;5(2):1362. doi: 10.4249/scholarpedia.1362

48. Caporale N, Dan Y. Spike timing-dependent plasticity: a Hebbian learning rule. Annu Rev Neurosci. 2008;31(1):25-46. doi: 10.1146/annurev.neuro.31.060407.125639 PMID: 18275283

49. Mikkelsen K, Imparato A, Torcini A. Emergence of slow collective oscillations in neural networks with spike-timing dependent plasticity. Phys Rev Lett. 2013;110(20):208101. doi: 10.1103/PhysRevLett.110.208101 PMID: 25167453

50. Bienenstock EL, Cooper LN, Munro PW. Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J Neurosci. 1982;2(1):32-48. doi: 10.1523/JNEUROSCI.02-01-00032.1982 PMID: 7054394

51. Lamsa K, Heeroma JH, Kullmann DM. Hebbian LTP in feed-forward inhibitory interneurons and the temporal fidelity of input discrimination. Nat Neurosci. 2005;8(7):916-24. doi: 10.1038/nn1486 PMID: 15937481

52. Lamsa KP, Heeroma JH, Somogyi P, Rusakov DA, Kullmann DM. Anti-Hebbian long-term potentiation in the hippocampal feedback inhibitory circuit. Science. 2007;315(5816):1262-1266.

53. Softky WR, Koch C. The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J Neurosci. 1993;13(1):334-50. doi: 10.1523/JNEUROSCI.13-01-00334.1993 PMID: 8423479

54. Ermentrout GB, Kopell N. Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J Appl Math. 1986;46(2):233-53. https://doi.org/10.1137/0146017

55. Perkel DH, Gerstein GL, Moore GP. Neuronal spike trains and stochastic point processes: II. Simultaneous spike trains. Biophys J. 1967;7(4):419-40. doi: 10.1016/S0006-3495(67)86597-4 PMID: 4292792

56. De Felipe J, Marco P, Fairén A, Jones E. Inhibitory synaptogenesis in mouse somatosensory cortex. Cereb Cortex. 1997;7(7):619-34. doi: 10.1093/cercor/7.7.619 PMID: 9373018

57. Rubinski A, Ziv NE. Remodeling and tenacity of inhibitory synapses: relationships with network activity and neighboring excitatory synapses. PLoS Comput Biol. 2015;11(11):e1004632. doi: 10.1371/journal.pcbi.1004632 PMID: 26599330

58. Karunakaran S, Chowdhury A, Donato F, Quairiaux C, Michel CM, Caroni P. PV plasticity sustained through D1/5 dopamine signaling required for long-term memory consolidation. Nat Neurosci. 2016;19(3):454-64. doi: 10.1038/nn.4231 PMID: 26807952

59. Barron HC, Vogels TP, Behrens TE, Ramaswami M. Inhibitory engrams in perception and memory. Proc Natl Acad Sci U S A. 2017;114(26):6666-74. doi: 10.1073/pnas.1701812114 PMID: 28611219

60. Mongillo G, Rumpel S, Loewenstein Y. Inhibitory connectivity defines the realm of excitatory plasticity. Nat Neurosci. 2018;21(10):1463-70. doi: 10.1038/s41593-018-0226-x PMID: 30224809

61. Giorgi C, Marinelli S. Roles and transcriptional responses of inhibitory neurons in learning and memory. Front Mol Neurosci. 2021;14:689952. doi: 10.3389/fnmol.2021.689952 PMID: 34211369
62. Arcelli P, Frassoni C, Regondi M, De Biasi S, Spreafico R. GABAergic neurons in mammalian thalamus: a marker of thalamic complexity? Brain Res Bull. 1997;42(1):27-37. doi: 10.1016/s0361-9230(96)00107-4 PMID: 8978932

63. Avino TA, Barger N, Vargas MV, Carlson EL, Amaral DG, Bauman MD, et al. Neuron numbers increase in the human amygdala from birth to adulthood, but not in autism. Proc Natl Acad Sci U S A. 2018;115(14):3710-5. doi: 10.1073/pnas.1801912115 PMID: 29559529

64. D'Angelo E. Physiology of the cerebellum. Handb Clin Neurol. 2018;154:85-108. doi: 10.1016/B978-0-444-63956-1.00006-0 PMID: 29903454

65. Gandolfi D, Mapelli J, Solinas SM, Triebkorn P, D'Angelo E, Jirsa V, et al. Full-scale scaffold model of the human hippocampus CA1 area. Nat Comput Sci. 2023;3(3):264-76. doi: 10.1038/s43588-023-00417-2 PMID: 38177882

66. Hardman CD, Henderson JM, Finkelstein DI, Horne MK, Paxinos G, Halliday GM. Comparison of the basal ganglia in rats, marmosets, macaques, baboons, and humans: volume and neuronal number for the output, internal relay, and striatal modulating nuclei. J Comp Neurol. 2002;445(3):238-55. doi: 10.1002/cne.10165 PMID: 11920704

67. Herrero MT, Barcia C, Navarro J. Functional anatomy of thalamus and basal ganglia. Childs Nerv Syst. 2002;18(8):386-404. doi: 10.1007/s00381-002-0604-1 PMID: 12192499

68. Karlsen AS, Pakkenberg B. Total numbers of neurons and glial cells in cortex and basal ganglia of aged brains with Down syndrome—a stereological study. Cereb Cortex. 2011;21(11):2519-24. doi: 10.1093/cercor/bhr033 PMID: 21427166

69. Pelkey KA, Chittajallu R, Craig MT, Tricoire L, Wester JC, McBain CJ. Hippocampal GABAergic inhibitory interneurons. Physiol Rev. 2017;97(4):1619-747. doi: 10.1152/physrev.00007.2017 PMID: 28954853

70. Sah P, Faber EL, Lopez de Armentia M, Power J. The amygdaloid complex: anatomy and physiology. Physiol Rev. 2003;83(3):803-34. doi: 10.1152/physrev.00002.2003 PMID: 12843409

71. Schröder K, Hopf A, Lange H, Thörner G. Morphometrical-statistical structure analysis of human striatum, pallidum and subthalamic nucleus. J Hirnforsch. 1975;16(4):333-50 PMID: 1214057

72. Singh-Bains MK, Waldvogel HJ, Faull RL. The role of the human globus pallidus in H untington's disease. Brain Pathol. 2016;26(6):741-51. doi: 10.1111/bpa.12429 PMID: 27529459

73. Spampanato J, Polepalli J, Sah P. Interneurons in the basolateral amygdala. Neuropharmacology. 2011;60(5):765-73. doi: 10.1016/j.neuropharm.2010.11.006 PMID: 21093462

74. Von Bartheld CS, Bahney J, Herculano-Houzel S. The search for true numbers of neurons and glial cells in the human brain: A review of 150 years of cell counting. J Comp Neurol. 2016;524(18):3865-95. doi: 10.1002/cne.24040 PMID: 27187682

75. Erö C, Gewaltig MO, Keller D, Markram H. A cell atlas for the mouse brain. Front Neuroinform. 2018;12:84. doi: 10.3389/fninf.2018.00084 PMID: 30546301

76. Rodarie D, Verasztó C, Roussel Y, Reimann M, Keller D, Ramaswamy S, et al. A method to estimate the cellular composition of the mouse brain from heterogeneous datasets. PLoS Comput Biol. 2022;18(12):e1010739. doi: 10.1371/journal.pcbi.1010739 PMID: 36542673

77. Roussel Y, Verasztó C, Rodarie D, Damart T, Reimann M, Ramaswamy S, et al. Mapping of morpho-electric features to molecular identity of cortical inhibitory neurons. PLoS Comput Biol. 2023;19(1):e1010058. doi: 10.1371/journal.pcbi.1010058 PMID: 36602951

78. McGaugh JL. The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annu Rev Neurosci. 2004;27:1-28. doi: 10.1146/annurev.neuro.27.070203.144157 PMID: 15217324

79. Rigotti M, Barak O, Warden MR, Wang XJ, Daw ND, Miller EK, et al. The importance of mixed selectivity in complex cognitive tasks. Nature. 2013;497(7451):585-90. doi: 10.1038/nature12160 PMID: 23685452

80. Parthasarathy A, Herikstad R, Bong JH, Medina FS, Libedinsky C, Yen SC. Mixed selectivity morphs population codes in prefrontal cortex. Nat Neurosci. 2017;20(12):1770-9. doi: 10.1038/s41593-017-0003-2 PMID: 29184197

81. Mi Y, Katkov M, Tsodyks M. Synaptic correlates of working memory capacity. Neuron. 2017;93(2):323-330. doi: 10.1016/j.neuron.2016.12.004 PMID: 28041884

82. Taher H, Torcini A, Olmi S. Exact neural mass model for synaptic-based working memory. PLoS Comput Biol. 2020;16(12):e1008533. doi: 10.1371/journal.pcbi.1008533 PMID: 33320855

83. Petilla terminology: nomenclature of features of GABAergic interneurons of the cerebral cortex. Nat Rev Neurosci. 2008;9(7):557-68. doi: 10.1038/nrn2402 PMID: 18568015

84. Kullmann DM, Moreau AW, Bakiri Y, Nicholson E. Plasticity of inhibition. Neuron. 2012;75(6):951-62. doi: 10.1016/j.neuron.2012.07.030 PMID: 22998865
85. Koch G, Ponzo V, Di Lorenzo F, Caltagirone C, Veniero D. Hebbian and anti-Hebbian spike-timing-dependent plasticity of human cortico-cortical connections. *J Neurosci.* 2013;33(23):9725-33. doi: 10.1523/JNEUROSCI.4988-12.2013 PMID: 23739969

86. Wu YK, Miehl C, Gjorgjieva J. Regulation of circuit organization and function through inhibitory synaptic plasticity. *Trends Neurosci.* 2022;45(12):884-98. doi: 10.1016/j.tins.2022.10.006 PMID: 36404455

87. Guy J, Möck M, Staiger JF. Direction selectivity of inhibitory interneurons in mouse barrel cortex differs between interneuron subtypes. *Cell Rep.* 2023;42(1):111936. doi: 10.1016/j.celrep.2022.111936 PMID: 36640357

88. Politi A, Torcini A. A robust balancing mechanism for spiking neural networks. *Chaos* 2024;34(4):041102. doi:10.1063/5.0199298

89. Földiák P. Forming sparse representations by local anti-Hebbian learning. *Biol Cybern.* 1990;64(2):165-170. doi: 10.1007/BF02331346 PMID: 2291903

90. Luz Y, Shamir M. Balancing feed-forward excitation and inhibition via Hebbian inhibitory synaptic plasticity. *PLoS Comput Biol.* 2012;8(1):e1002334. doi: 10.1371/journal.pcbi.1002334 PMID: 22291583

91. Kleberg FI, Fukai T, Gilson M. Excitatory and inhibitory STDP jointly tune feedforward neural circuits to selectively propagate correlated spiking activity. *Front Comput Neurosci.* 2014;8:53. doi: 10.3389/fncom.2014.00053 PMID: 24847242

92. Bliss TV, Collingridge GL. A synaptic model of memory: long-term potentiation in the hippocampus. *Nature*. 1993;361(6407):31-9. doi: 10.1038/361031a0 PMID: 8421494

93. McGaugh JL. Memory-a century of consolidation. *Science*. 2000;287(5451):248-51. doi: 10.1126/science.287.5451.248 PMID: 10634773

94. Malenka RC, Bear MF. LTP and LTD: an embarrassment of riches. *Neuron*. 2004;44(1):5-21. doi: 10.1016/j.neuron.2004.09.012 PMID: 15450156

95. Torao-Angosto M, Manasanch A, Mattia M, Sanchez-Vives MV. Up and down states during slow oscillations in slow-wave sleep and different levels of anesthesia. *Front Syst Neurosci.* 2021;15:609645. doi: 10.3389/fnsys.2021.609645 PMID: 33633546

96. Wang JY, Heck KL, Born J, Ngo HVV, Diekelmann S. No difference between slow oscillation up-and down-state cueing for memory consolidation during sleep. *J Sleep Res.* 2022;31(6):e13562. doi: 10.1111/jsr.13562 PMID: 35166422

97. Squire LR, Alvarez P. Retrograde amnesia and memory consolidation: a neurobiological perspective. *Curr Opin Neurobiol.* 1995;5(2):169-77. doi: 10.1016/0959-4388(95)80023-9 PMID: 7620304

98. Steriade M. Impact of network activities on neuronal properties in corticothalamic systems. *J Neurophysiol.* 2001;86(1):1-39. doi: 10.1152/jn.2001.86.1.1 PMID: 11431485

99. Stickgold R. Sleep-dependent memory consolidation. *Nature* 2005;437:1272-8. https://doi.org/10.1038/nature04286

100. Marzano C, Ferrara M, Mauro F, Moroni F, Gorgoni M, Tempesta D, et al. Recalling and forgetting dreams: theta and alpha oscillations during sleep predict subsequent dream recall. *J Neurosci.* 2011;31(18):6674-83. doi: 10.1523/JNEUROSCI.0412-11.2011 PMID: 21543596

101. Dudai Y. The restless engram: consolidations never end. *Annu Rev Neurosci.* 2012;35:227-47. doi: 10.1146/annurev-neuro-062111-150500 PMID: 22443508

102. Theodoni P, Rovira B, Wang Y, Roxin A. Theta-modulation drives the emergence of connectivity patterns underlying replay in a network model of place cells. *Elife*. 2018;7:e37388. doi: 10.7554/eLife.37388 PMID: 30355442

103. Eichenlaub JB, Nicolas A, Daltrozzo J, Redouté J, Costes N, Ruby P. Resting brain activity varies with dream recall frequency between subjects. *Neuropsychopharmacology*. 2014;39(7):1594-602. doi: 10.1038/npp.2014.6 PMID: 24549103

104. Buzsaki G, Draguhn A. Neuronal oscillations in cortical networks. *Science*. 2004;304(5679):1926-9. doi: 10.1126/science.1099745 PMID: 15218136

105. Fries P. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. *Trends Cogn Sci.* 2005 Oct;9(10):474-80. doi: 10.1016/j.tics.2005.08.011 PMID: 16150631

106. Rolls ET, Deco G. The noisy brain: stochastic dynamics as a principle of brain function. *Oxford University Press*; 2010.

107. Hopfield JJ. Neural networks and physical systems with emergent collective computational abilities. *Proc Natl Acad Sci U S A.* 1982;79(8):2554-8. doi: 10.1073/pnas.79.8.2554 PMID: 6953413
108. Amit DJ, Gutfreund H, Sompolinsky H. Storing infinite numbers of patterns in a spin-glass model of neural networks. Phys Rev Lett. 1985;55(14):1530-3. doi: 10.1103/PhysRevLett.55.1530 PMID: 10031847

109. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. Invariant visual representation by single neurons in the human brain. Nature. 2005;435(7045):1102-7. doi: 10.1038/nature03687 PMID: 15973409

110. Johnston WJ, Palmer SE, Freedman DJ. Nonlinear mixed selectivity supports reliable neural computation. PLoS Comput Biol. 2020;16(2):e1007544. doi: 10.1371/journal.pcbi.1007544 PMID: 32069273

111. Bonifazi P, Goldin M, Picardo MA, Jorquera I, Cattani A, Bianconi G, et al. GABAergic hub neurons orchestrate synchrony in developing hippocampal networks. Science. 2009;326(5958):1419-24. doi: 10.1126/science.1175509 PMID: 19965761

112. Van Den Heuvel MP, Sporns O. Rich-club organization of the human connectome. J Neurosci. 2011;31(44):15775-86. doi: 10.1523/JNEUROSCI.3539-11.2011 PMID: 22049421

113. Mountcastle VB. Modality and topographic properties of single neurons of cat's somatic sensory cortex. J Neurophysiol. 1957;20(4):408-34. doi: 10.1152/jn.1957.20.4.408 PMID: 13439410

114. Kaas JH. Topographic maps are fundamental to sensory processing. Brain Res Bull. 1997;44(2):107-12. doi: 10.1016/s0361-9230(97)00094-4 PMID: 9292198

115. Kaas JH, Collins CE. The organization of sensory cortex. Curr Opin Neurobiol. 2001;11(4):498-504. doi: 10.1016/s0959-4388(00)00240-3 PMID: 11502398

116. Petreanu L, Mao T, Sternson SM, Svoboda K. The subcellular organization of neocortical excitatory connections. Nature. 2009;457(7233):1142-5. doi: 10.1038/nature07709 PMID: 19151697

117. Zhang S, Xu M, Kamigaki T, Hoang Do JP, Chang WC, Jenvay S, et al. Long-range and local circuits for top-down modulation of visual cortex processing. Science. 2014;345(6197):660-5. doi: 10.1126/science.1254126 PMID: 25104383

118. Caputi A, Melzer S, Michael M, Monyer H. The long and short of GABAergic neurons. Curr Opin Neurobiol. 2013;23(2):179-86. doi: 10.1016/j.conb.2013.01.021 PMID: 23394773

119. Gerstner W, Kistler WM, Naud R, Paninski L. Neuronal dynamics: from single neurons to networks and models of cognition. Cambridge University Press; 2014.

120. Maass W. On the computational power of winner-take-all. Neural Comput. 2000;12(11):2519-35. doi: 10.1162/089976600300014827 PMID: 11110125

121. Abeles M. Corticonics: neural circuits of the cerebral cortex. Cambridge University Press; 1991.

122. Toral R, Colet P. Stochastic numerical methods: an introduction for students and scientists. John Wiley & Sons; 2014.

123. Tallon-Baudry C, Bertrand O, Delpuech C, Pernier J. Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J Neurosci. 1996;16(13):4240-9. doi: 10.1523/JNEUROSCI.16-13-04240.1996 PMID: 8753885

124. Herrmann CS. Human EEG responses to 1-100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp Brain Res. 2001;137(3-4):346-53. doi: 10.1007/s002210100682 PMID: 11355381

125. Carlson KD, Richert M, Dutt N, Krichmar JL. Biologically plausible models of homeostasis and STDP: stability and learning in spiking neural networks. In: The 2013 International Joint Conference on Neural Networks (IJCNN). IEEE; 2013. pp. 1-8.

126. Wixted JT. The psychology and neuroscience of forgetting. Annu Rev Psychol. 2004;55:235-69. doi: 10.1146/annurev.psych.55.090902.141555 PMID: 14744216

127. Hardt O, Nader K, Nadel L. Decay happens: the role of active forgetting in memory. Trends Cogn Sci. 2013;17(3):111-20. doi: 10.1016/j.tics.2013.01.001 PMID: 23369831

128. Perez Y, Morin F, Lacaille JC. A hebbian form of long-term potentiation dependent on mGluR1a in hippocampal inhibitory interneurons. Proc Natl Acad Sci U S A. 2001;98(16):9401-6. doi: 10.1073/pnas.161493498 PMID: 11447296

129. Plumbley MD. Efficient information transfer and anti-Hebbian neural networks. Neural Networks. 1993;6(6):823-33. https://doi.org/10.1016/S0893-6080(05)80127-3

130. Jin Y, Li P. AP-STDP: A novel self-organizing mechanism for efficient reservoir computing. In: 2016 International Joint Conference on Neural Networks (IJCNN). IEEE; 2016. pp. 1158-65.

131. Van Rossum MC, Shippi M, Barrett AB. Soft-bound synaptic plasticity increases storage capacity. PLoS Comput Biol. 2012;8(12):e1002836. doi: 10.1371/journal.pcbi.1002836 PMID: 23284281

132. Gilson M, Fukai T. Stability versus neuronal specialization for STDP: long-tail weight distributions solve the dilemma. PLoS One. 2011;6(10):e25339. doi: 10.1371/journal.pone.0025339 PMID: 22003389

133. Kuramoto Y. Chemical oscillations, waves, and turbulence. Courier Corporation; 2003.