Title: Meta-learning families of plasticity rules in recurrent spiking networks using simulation-based inference
Venue: NeurIPS 2023
Accepted: yes
Keywords: synaptic plasticity

spiking network

meta-learning

computational neuroscience
Abstract: There is substantial experimental evidence that learning and memory-related behaviours rely on local synaptic changes, but the search for distinct plasticity rules has been driven by human intuition, with limited success for multiple, co-active plasticity rules in biological networks. More recently, automated meta-learning approaches have been used in simplified settings, such as rate networks and small feed-forward spiking networks. Here, we develop a simulation-based inference (SBI) method for sequentially filtering plasticity rules through an increasingly fine mesh of constraints that can be modified on-the-fly. This method, _filter SBI_, allows us to infer entire families of complex and co-active plasticity rules in spiking networks. We first consider flexibly parameterized doublet (Hebbian) rules, and find that the set of inferred rules contains solutions that extend and refine -and also reject- predictions from mean-field theory. Next, we expand the search space of plasticity rules by modelling them as multi-layer perceptrons that combine several plasticity-relevant factors, such as weight, voltage, triplets and co-dependency. Out of the millions of possible rules, we identify thousands of unique rule combinations that satisfy biological constraints like plausible activity and weight dynamics. The resulting rules can be used as a starting point for further investigations into specific network computations, and already suggest refinements and predictions for classical experimental approaches on plasticity. This flexible approach for principled exploration of complex plasticity rules in large recurrent spiking networks presents the most advanced search tool to date for enabling robust predictions and deep insights into the plasticity mechanisms underlying brain function.

Meta-learning families of plasticity rules in recurrent spiking networks using simulation-based inference

Abstract

There is substantial experimental evidence that learning- and memory-related behaviours rely on local synaptic changes, but the search for distinct plasticity rules has been driven by human intuition, with limited success for multiple, co-active plasticity rules in biological networks. More recently, automated meta-learning approaches have been used in simplified settings, such as rate networks and small feed-forward spiking networks. Here, we develop a simulation-based inference (SBI) method for sequentially filtering plasticity rules through an increasingly fine mesh of constraints that can be modified on-the-fly. This method, filter SBI , allows us to infer entire families of complex and co-active plasticity rules in spiking networks. We first consider flexibly parameterized doublet (Hebbian) rules, and find that the set of inferred rules contains solutions that extend and refine-and also reject-predictions from mean-field theory. Next, we expand the search space of plasticity rules by modelling them as multi-layer perceptrons that combine several plasticity-relevant factors, such as weight, voltage, triplets and co-dependency. Out of the millions of possible rules, we identify thousands of unique rule combinations that satisfy biological constraints like plausible activity and weight dynamics. They can be used as a starting point for further investigations into specific network computations, and already suggest refinements and predictions for classical experimental approaches on plasticity. This flexible approach for principled exploration of complex plasticity rules in large recurrent spiking networks presents the most advanced search tool to date for enabling robust predictions and deep insights into the plasticity mechanisms underlying brain function.

1 Introduction and related work

Synaptic plasticity has received continuous attention over the past decades [1-5]. Nevertheless, how functions such as learning and memory emerge from local synaptic changes is still poorly understood. Given the experimental inaccessibility of synapses in vivo , many studies have relied

on theoretical methods to shed light on the plasticity rules at play [6-12]. However, this approach does not scale [3, 4]-in particular, previous attempts at tuning complex, co-active plasticity rules have been limited by human intuition, and had to rely on theoretical frameworks based on strongly simplifying assumptions [8, 9].

Recently, several studies have aimed to address these issues by automating the discovery of plasticity rules using numerical meta-learning approaches [13-20]. Given the non-differentiability of biological systems, along with the steep compute requirements of simulating plastic networks at scale, these studies have been restricted to rate networks [13-16, 18-20] or small feed-forward spiking networks [15, 17]. Moreover, these approaches require the a priori definition and optimization of a loss function that enforces a desired network computation, as well as various regularizers to ensure the implementation of the task is biologically plausible. Lastly, the above studies typically propose single plasticity mechanisms compatible with the data or network function considered.

Here, we aim at a more comprehensive approach to discovering biologically plausible plasticity rules. We study network dynamics in large recurrent spiking networks with plasticity rules that are flexibly parameterized (using polynomials or neural networks, Fig. 1A). We introduce filter simulation-based inference (fSBI), a new meta-learning approach that can thoroughly explore the space of potential rules. This method allows us to select sets of rules using metrics that effectively constrain network dynamics to fulfill the desired conditions. Our approach successfully infers entire families of rules that robustly establish plausible dynamics in large spiking networks. We demonstrate the scalability of fSBI by applying it to neural-network-based search spaces that include additional factors known to influence plasticity, such as, e.g., synaptic weight or membrane potentials. Despite the nonlinear interactions induced by the neural network parameterization of plasticity and the increase in the number of factors, fSBI successfully recovers plausible rules. Interestingly, when considered in isolation, the inferred candidate rules often result in contradictory experimental predictions, revealing that the classical pre-post protocols may only have marginal explanatory power.

Overall, our study introduces a principled approach for meta-learning plasticity rules in complex networks. The inferred distributions of plausible rules can be used as starting points for future studies of specific network computations, and raise questions about classical theoretical and experimental protocols for understanding plasticity.

A

Plasticrecurrent spiking network

B

Prior

Filtered posterior 1

Filteredposterior2

To(0)

p(θ|△w) =π2(0)

Parameterized

plasticityrules

△w=f(localvars,θ)

refine

refine

Polynomial

MLP

Iterativelynarrow-downthesearch-space

2 Methods

2.1 Spiking network model

We consider a recurrent spiking network of 4096 excitatory and 1024 inhibitory leaky integrate-andfire neurons with conductance-based synapses, relative refractoriness, adaptive threshold and NMDA currents. The model parameter values follow Zenke et al. [8] (Fig. 1A; see Supp. 6.1 for a complete description of parameters). The membrane potential dynamics of neuron j (excitatory or inhibitory) are given by

where E stands for excitation and I for inhibition. A postsynaptic spike is emitted whenever the membrane potential V j ( t ) crosses a threshold V th j ( t ) , with an instantaneous reset to V reset . This threshold V th j ( t ) is incremented by V th spike every time neuron j spikes and otherwise decays following

The excitatory and inhibitory conductances, g E and g I evolve such that

with w ij ( t ) the connection strength between neurons i and j (unitless), δ k ( t ) = ∑ δ ( t -t ∗ k ) the spike train of pre-synaptic neuron k , where t ∗ k denotes the spike times of neuron k , and δ the Dirac delta. All neurons received input from 5k Poisson neurons, with 5% random connectivity and constant rate r ext, randomly sampled from the uniform distribution U (5 , 15) Hz in each simulation. The recurrent connectivity was instantiated with random sparse connectivity (10%).

2.2 Synaptic plasticity parameterizations

In order to explore a wide range of possible plasticity rules, we defined parameterized functional forms for these rules (i.e., search spaces) enabling us to flexibly generate various rules, by changing the parameters of the function. We considered two such parameterizations:

For polynomial plasticity rules , each recurrent synapse underwent spike-timing dependent plasticity (STDP) [21]. The weight from neuron i to j (of type X and Y; ( X , Y ) ∈ { E, I } ) evolved such that

with η a fixed learning rate. The variables x i ( t ) and x j ( t ) trace the pre- and post-synaptic spike trains

where τ pre XY and τ post XY are the time constants of the traces associated with the pre- and postsynaptic neurons, respectively. Thus, the plasticity updates at each connection type depended on 6 parameters: θ XY = [ α XY , β XY , γ XY , κ XY , τ pre XY , τ post XY ], for a total of 24 parameters across all connection types (Fig. 2A).

For the MLP plasticity rules , only the recurrent EE and IE synapses underwent spike-triggered updates (Fig. 4A). Change of the weight from neuron i to j (of type X and E, X ∈ (E, I)) was computed by running forward multi-layer perceptrons (MLPs) that took as inputs several synaptic variables relevant to plasticity 1 :

with C exc j = ⟨ g E j ( E E -V j ) ⟩ and C inh j = ⟨ g I j ( E I -V j ) ⟩ co-dependent terms representing the activity of neighboring synapses [10] (here all synapses with the same postsynaptic neuron), and ⟨ V j ( t ) ⟩ the low-pass filtered membrane potential [11]. In this search space, time constants of synaptic traces were fixed at τ (1) EE = τ (1) IE = 10 ms and τ (2) EE = τ (2) IE = 100 ms [22, 7, 8, 10]. We chose a two-hidden-layer MLP, composed of 50 sigmoidal units in the first hidden layer, and 4 in the second. We trained only the final layer, keeping all other layers fixed. While this is standard practice with pre-trained ML

models, we randomly initialised the parameters of the frozen layers, rather than pre-training them. We verified that when using random initializations for the first 2 hidden layers ( ∼ U ( -1 n inp , 1 n inp ) , where n inp is the number of input features at a given layer), and only training the weights, bias, and output learning rate of the final layer, previously observed plasticity rules could be approximated (Supp. 6.7).

A given plasticity rule (EE or IE) thus consisted of one MLP (with 5 parameters: 4 weights and 1 bias) to update the synaptic weight when a pre-spike occurred, another MLP updating the synaptic weight for post-spikes (Fig. 4B), and a common learning rate η XE for both MLPs, resulting in 11 parameters per rule, and a total of 22 parameters between EE and IE.

2.3 Mean-Field predictions

In the polynomial case, we performed a self-consistent analysis of the weight and activity dynamics in the network at steady state, following previous work [23, 7, 10] (more details for all connection types in Supp. 6.3). For example, at EE synapses, we could calculate an expected firing rate r ∗ exc from the parameters of the rule such that:

with λ EE = κ EE τ post EE + γ EE τ pre EE

2.4 Assessing network plausibility with metrics

To assess whether a particular plasticity rule was a suitable candidate, we simulated a spiking neuronal network (SNN) with that rule and quantified the resulting dynamics with multiple metrics on the activity and weight traces. The metric choices and values were informed by-and in line with-a range of modeling and experimental cortical studies [7, 8, 24-34]. For every simulation, we stored the raw spike trains and weight traces for 10s after 110s of simulated time (60s for the MLP rules). We grouped 15 individual metrics into four main criteria:

Stable activity dynamics : Rules were flagged as suitable when the excitatory and inhibitory population firing rates were bounded between 1 and 50 Hz, i.e., r exc ∈ [1 , 50] Hz and r inh ∈ [1 , 50] Hz, in line with a large swathe of literature reporting cortical firing rates of a few Hertz on average [24, 29, 30].

Stable weight dynamics : Fraction of synaptic weights that reached extreme values ( 0 or w max), f w blow ; mean absolute change in synaptic weights between simulation start and end across all synapse types, w creep; mean weight at the last time-step across all synapses of a specific type, ⟨ w XY ⟩ . These metrics excluded rules whose weights changed too rapidly, did not converge, or reached unrealistic values. Rules were flagged as suitable if they satisfied f w blow < 0 . 1 , w creep < 0 . 05 , ⟨ w EX ⟩ < 0 . 5 and ⟨ w IX ⟩ < 5 simultaneously.

Near-irregular dynamics : Mean coefficient of variation of the inter-spike intervals across all excitatory neurons, ⟨ cv ( ISI i ) ⟩ ; auto-covariance for each spike-train, and its integral, averaged over time and the neural population, ⟨ ρ ⟩ i,t ; Fano factor for each spike train averaged over the population, ⟨ Fano ⟩ i ; standard deviation of the firing rate of individual neurons averaged across the population, ⟨ σ ⟩ i . These metrics quantified the regularity of a spike train, but, either due to metric assumptions or hyperparameter choices, each metric exhibited a different failure mode, which could be filtered out by combining the metrics. Rules were flagged as suitable if they satisfied ⟨ cv ( ISI i ) ⟩ > 0 . 7 , ⟨ ρ ⟩ i,t < 0 . 1 , ⟨ Fano ⟩ i ∈ [0 . 5 , 2 . 5] and ⟨ σ ⟩ i < 5 simultaneously.

Near-asynchronous dynamics : Standard deviation of the excitatory population firing rate, σ r exc ; the Fano factor averaged across the population and then across time windows, ⟨ Fano ⟩ t ; we also computed the population average of binned spike-trains, Fourier-transformed it and averaged the resulting power spectrum across all frequencies f (except 0), < S > i,f . Rules were flagged as suitable if they satisfied ⟨ Fano ⟩ t ∈ [0 . 5 , 2 . 5] , < S > i,f < 1 and σ r exc < 0 . 05 simultaneously. These bounds enforced the broadly asynchronous-irregular activity observed in cortex [24, 25, 27, 29, 30]. Ranges for metrics assessing synchrony/regularity were thus devised with independent Poisson spike trains in mind, with leeway for spatiotemporal correlations, as seen in cortical recordings [32-34].

Rules were considered plausible if they fulfilled all four broad criteria above (more details and analysis of individual metrics in Supp. 6.2).

2.5 Filter simulation-based inference

Having defined plasticity rules search spaces and network metrics, we turned to the problem of exploring these spaces, systematically filtering out unsuitable rules.

Simulation-based inference (SBI) is broadly defined as an approach for performing Bayesian inference given stochastic models (with tunable parameters θ , and generating samples x ) in cases where the likelihood of the model p ( x | θ ) is analytically inaccessible but simulation is possible [3537]. SBI infers the posterior distribution p ( θ | x ) , using samples from a prior distribution θ ∼ π ( θ ) and corresponding simulations x ∼ p ( x | θ ) , i.e., it applies Bayes' rule such that p ( θ | x ) ∝ p ( x | θ ) π ( θ ) , where p ( θ | x ) represents the distribution over model parameters most likely to have generated x .

This is an appealing framework for the problem of identifying plausible plasticity rules: we treated the parameters of the plasticity rule as parameters θ of a stochastic plastic spiking network that generated simulations x consisting of the metrics defined in Section 2.4. We then used SBI-e.g., Neural Posterior Estimation (NPE) [35, 38, 39]-to identify the posterior distribution over plasticity rule parameters that were most likely to have resulted in plausible network dynamics. However, as we could not know beforehand which parameters were most likely to establish plausible dynamics, we had to define a broad prior over the parameters of the plasticity rule π 0 ( θ ) . In turn, the broadness of the prior meant that potentially millions of SNN simulations would be needed to find a few plausible rules, rendering this search method computationally unfeasible [40, 41].

In order to address this issue, we developed filter SBI (fSBI) . In fSBI, we applied the metrics quantifying plausibility sequentially , rather than all at once (Fig. 1B). Towards this goal, we first sampled plasticity rule parameters θ from a uniform prior and simulated an SNN with the corresponding rules. Next, we computed the various metrics defined in Section 2.4. We then used NPE to fit a posterior density over the plasticity rule parameters. Finally, we sampled all parameters that satisfied a given criterion from the posterior, e.g., stable network activity p ( θ | r exc ∈ [1 , 50] Hz ∩ r inh ∈ [1 , 50] Hz ) , in the first instance, effectively "filtering out" other rules from the prior. These posterior samples were then used as the prior samples for the next round of inference, in which an additional criterion was applied to obtain a new, narrower posterior. Mathematically speaking, a filtering round k using metrics m k = f k ( x ) and corresponding conditions g k ( m k ) proceeds as follows:

where I is an indicator function over the condition g . This process is repeated until only plausible candidate rules remain, for a total of around 200k simulated rules per search space.

The distribution obtained at each filtering round is not necessarily a true Bayesian posterior, since (a) we change the metric on which we condition the posterior at each round, (b) we sample the posterior for a range of metric values rather than a single value, and (c) we also do not correct for the fact that at each round, we are sampling from the posterior of the previous round rather than from the prior [42] (more detailed analysis of this "pseudo"-posterior in Supp. 6.5).

3 Results

In our attempt to discover plasticity rules compatible with a given function, we introduced fSBI, a meta learning method that sequentially filters out unfit candidates rules from vast sets of possible plasticity rules in large recurrent spiking networks (SNNs, Methods. 2.1,10). We first applied fSBI in the polynomial search space described in Section 2.2, and compared the resulting distributions of rules to those predicted by mean-field analysis. We then applied fSBI to plasticity rules which themselves were parameterized by neural networks. In both cases, fSBI inferred expected and novel rules that robustly establish plausible dynamics in SNNs, while also eliminating some theoretically viable solutions.

3.1 fSBI infers plausible rules from a polynomial search space

We initially considered 4 co-active plasticity rules, E-to-E (EE), E-to-I (EI), I-to-E (IE) and I-toI (II) in a recurrent spiking network of 5120 conductance-based leaky integrate-and-fire neurons (Fig. 2A). The network architecture and parameters were chosen following Zenke et al. [8]. Each

A △wxr(B) = Poly(xpre,xpost)

B

Stable weights

fraction of rules

sampled from

1

- stable activity

A.l.

10-1,

Stable

10-2,

－stable weights

Inh

activity

10-3.

A.l. dynamics

△WIE(O)

>

10-4

-plausible

rext

To

T3

TTo

T1

T2

T3

D

E

H

Example plausible rule from T3

explained

variance

0.10

1

rext =5Hz

rext = 15Hz

△WEE

To

2'!(d)

0.2

0.05

T1

△t (s)

TT2

Inh

Inh

-0.2

0.2

0.00+

T3

0

rg (Hz)

θdimensions

1

24

0

(cv(ISI;));

1

2

10

10

△t (s)

F

G

0

neurons

0.1

-Wee -Wei

0.1

-Wee -Wei

△WIE

0.2

找

0

0

△t (s)

21

=Wie =Wii

2七

-Wie-Wii

10

0.5

0.5

100ms

100 ms

55

t (s)

60

55

t(s)

60

△t (s)

rule was parameterized as a polynomial (Eqn 6, [15]), with a total of 24 parameters θ (6 parameters per connection type). We applied fSBI to this 24-dimensional search space, to identify and retain the rules that established biologically plausible dynamics in SNNs. To ensure the inferred rules robustly enforced plausible dynamics in SNNs, any simulation was assigned a random input rate and connectivity matrix.

We started fSBI with uniform priors for all plasticity parameters: τ pre XY , τ pre XY ∼ U [10 , 100] ms, and α XY , β XY , γ XY , κ XY ∼ U [ -2 , 2] . The resulting rules formed the initial prior π 0 . We then sampled multiple plasticity rules θ ∼ π 0 , simulated SNNs with these plasticity rules, and saved their spiketrains and weight evolutions for offline analysis of the network metrics x (see Section 2.4). For the first round, we trained Neural Posterior Estimation (NPE) to produce posteriors conditioned only on a subset of metrics-the excitatory and inhibitory population firing ratesp ( θ | r exc , r inh ) . Next, we sampled plasticity rules from this posterior corresponding to networks with excitatory and inhibitory rates between 1 and 50 Hz, i.e., θ ′ ∼ p ( θ ′ | r exc ∈ [1 , 50] Hz , r inh ∈ [1 , 50] Hz ) . These rules were used for the next round of inference, which we conditioned on the metrics associated with stable weight dynamics ( f w blow , w creep , ⟨ w XY ⟩ ), in addition to the population activity rates. We thus made the posterior from the previous iteration, p ( θ | r exc ∈ [1 , 50] Hz , r inh ∈ [1 , 50] Hz ) , into the prior π 1 ( θ ) of the current iteration. We progressively applied all above mentioned metrics (defined in Section 2.4), in the order of the least to the most restrictive metric, to maximize sample efficiency (Fig. 2B, C).

We quantified the behaviour of the network with a range of metrics [23, 25] and noticed that some networks satisfied some conditions, but not others which should discern similar qualities. For example, some networks passed the auto-covariance criterion ( ⟨ ρ ⟩ i,t < 0 . 1 ) but not the CV criterion ( ⟨ cv ( ISI i ) ⟩ > 0 . 7 ), by allowing a small population of neurons with unrealistically high firing rates to "cheat" the metric (Fig. 2E, F). Conversely, other networks passed the CV condition but failed the auto-covariance one, e.g., with a bimodal inter-spike interval (ISI) distribution (Fig. 2G). To

filter out these edge cases, we used several metrics for each desired aspect of the network dynamics (Section 2.4).

We found that fSBI successfully identified relevant parts of the search space: the 10,000 samples from the uniform prior π 0 ( θ ) contained no plausible rules (i.e., no rules producing networks satisfying all criteria). However, by sequentially adding constraints we arrived at a posterior π 3 ( θ ) that, at the end of the fourth search round, led to more than 50% suitable rules (Fig. 2C). Interestingly, the dimensionality of the posterior, i.e., of the 'suitable rule space", shrunk as we sequentially refined the criteria for plausible network dynamics (Fig. 2D), suggesting that fSBI captured specific relationships between the plasticity parameters. On the other hand, the final posterior remained relatively high dimensional, comprising a variety of different rules, with no apparent posterior collapse onto a single rule. The plausible rules sampled from this final posterior showed stable activity and weight dynamics, regardless of their input rates (Fig. 2H).

Interdependencies between plasticity parameters in the fSBI inferred rules : Next, we examined interdependencies between fSBI-inferred parameters as a possible explanation for the reduced effective dimensionality of the parameter space (Fig. 2D). We found that the correlation was overall weak between the parameters of the plausible rules (Supp. Fig. 6.2B). This was not surprising, as we compared a range of rules that establish a wide diversity of SNN dynamics, such that individual rules can be expected to have different parameter inter-dependencies. We thus narrowed our investigation to rules that produced excitatory firing rates of 10Hz (as opposed to a range of rates between 1 and 50Hz, Fig. 3A). The corresponding correlation matrix revealed a fine-grained structure for parameters within the same rule (block diagonal structure), where we observed mostly negative correlations, especially between the non-Hebbian parameters within each rule, α XY and β XY. Additionally, we found both positive and negative correlations between any given connection types (off-diagonal structure, Fig. 3A), suggesting that plasticity mechanisms must be co-tuned to function properly.

fSBI rules improve upon mean-field predictions : We also compared the fSBI-inferred rules to mean-field predictions. Mean-field analysis links the plasticity rule parameters to the average expected firing rate (Section 2.3, Fig. 3B), thus allowing us to obtain theoretically viable rules. Samples from the initial prior did not show any similarity to theoretical predictions when projected along the mean-field axes (Fig. 3C, left). As rules were filtered by progressively more constraints, a similar structure to the mean-field predictions emerged, albeit confined to firing rates below 10Hz, possibly because of synchronisation effects for higher firing rates (Fig. 3C for EE; similar conclusions for the other three synapse types in Supp. Fig.6.2). Interestingly, the set of rules predicted by mean field theory was not a superset of those selected by fSBI: Some rules were exclusively predicted by one but not the other approach (Fig. 3D, E). Notably, many rules predicted by mean-field, but not by fSBI, did not result in plausible dynamics when tested numerically (Fig. 3D, middle). In contrast, several rules predicted by fSBI but rejected by mean-field proved numerically viable (Fig. 3E, middle). Finally, the EE rules predicted by both mean-field and fSBI were plausible (Fig. 3D, bottom). We noted that fSBI rules lead to plausible network dynamics both when simulated with only the EE parameters (keeping the weights of the other connection types fixed, Fig. 3E, bottom), and also when using the rule for all connection types (Fig. 3E, middle), thus confirming the success of fSBI.

Comparing rules under pre-post protocols : We visualized the fSBI rules under the pre-post pairing protocol widely used experimentally and theoretically to uniquely identify STDP rules [1, 6, 43, 44]. We found that among the various rules simulated, there was no apparent link between how rules appeared in the pre-post protocol, and their ability to establish plausible dynamics in SNNs. Some rules appeared near-identical under the pre-post visualization and yet resulted in vastly different network dynamics (Fig. 3D). Conversely, rules with visibly differing pre-post protocols established similar network dynamics (Fig. 2H and Fig. 3E). Taken together, these results question the relevance of such classical pre-post protocols in elucidating and uniquely determining the plasticity rules at play in the brain, particularly when they are performed without additional disambiguating experiments.

3.2 fSBI infers plausible rules from an MLP search space

We wanted to expand our search space to include local and semi-local variables that have been theoretically or experimentally linked to synaptic changes [2-4], such as the efficacy of the synapse [44, 8], the post-synaptic membrane potential [11], the spike-history of both the pre- and post-synaptic neurons [22], as well as the activity of neighboring excitatory and inhibitory synapses (co-dependent plasticity [10]). Since flexibly combining these variables with a polynomial would result in very

A

T(0lrexc = 10Hz)

D

Mean-field posterior (EE)

E

fSBI posterior (π3)

Tpost

e

众

high

distance to

21

MF posterior

-2

-2

Tpost

-2

αEE

low

-2

αEE

low

cov(0, 0)

Fullrule

Tpost

Tpre

K

0

△WEE

-0.2

☆

0.2

10°

neurons

..

0.2

△t(s)

100 ms

100 ms

Tpost

Tpre

K

-0.2

0.2

Only EE

24

△WIE

β

Tpreα

Tpreα

Tpreα

y

-1

△WEE

△t(s)

10

neurons

?

..

:.

"

35

A

No.2

K Tpostβ

100 ms

100 ms

△t(s)

B Mean-field predictions

C

All rules

Stableactivity

→

Stableweights

Plausible

0.41

30

0.4 1.

30

0.41

T

30

0.41

『30

0.41

T

30

15

入EE

15

15

15

15.

4

4

4

4

4

-0.41

-αEE-βEE

=

0

-0.4

-αEE-βEE

0

-0.4

-αEE-βEE

0

-0.4

-αEE-βEE

0

-0.41

-αEE-βEE

0

A : Pearson correlation matrix of plasticity rules sampled from the final fSBI posterior ( π 3 ), conditioned on the excitatory firing rate r exc = 10 Hz. B : EE rules predicted by mean-field, plotted alongside the relevant parameter combinations (eq. 7). C : Rules sampled from fSBI posteriors, with an increasing number of conditions, along the same axes as in B. D : Same set of rules as in B, but shown along 2 of the 6 plasticity parameters ( α and β ), color coded by the mean Euclidean distance between each rule and the 10 closest fSBI-inferred EE rules. Middle (resp. bottom), network dynamics with a rule among the furthest (resp. closest) from fSBI samples, and the corresponding pre-post protocol. E : Rules sampled from the final fSBI posterior π 3 , shown along the same axes as D, color coded by the mean Euclidean distance between each rule and the 10 closest mean-field plasticity rules. Below, network dynamics with one of the rules furthest from mean-field, and the corresponding pre-post protocol, either the full rule (EE, EI, IE, II, middle) or only the EE part of the rule (bottom). Weight traces in Supp. 6.6.

high-dimensional, hard-to-navigate search spaces, we switched to multi-layer perceptrons (MLP). The MLP-based rules we designed used the aforementioned synaptic variables as inputs at the time of a spike, and produced a weight change for a given synapse (Fig. 4B), thus representing the plasticity rule. Crucially, we only trained the final layer of that MLP, thus allowing for potentially highly non-linear interactions between the plasticity variables while retaining a relatively low dimensional (searchable) parameter space. We verified that such MLP-parameterized rules can approximate established plasticity rules on supervised tasks (see Supp. Fig.6.6), and then applied fSBI to inferring co-active EE and IE rules (Fig. 4A, B). Like for the polynomial space, we found that fSBI produced progressively more constrained posteriors, such that the final fSBI posterior generated over 50% plausible rules (Fig. 4C). Furthermore, fSBI reduced the effective dimensionality of the parameter space, i.e., increased correlations between the parameters (Supp.6.7), like what we had observed for the polynomial expression of rules. Interestingly, plausible rules from the final fSBI posterior produce stable activity and weight dynamics, but with unique pre-post protocol profiles that are markedly different from that of experimentally observed rules, as well as the rules discovered by the polynomial method above (Fig. 4D, E, Supp.6.7). These rules represent potentially complex and non-linear interactions between Hebbian, local and semi-local factors, but nevertheless robustly establish plausible network dynamics.

Generalization capabilities of meta-learned rules : Finally, we verified that both the polynomial and the MLP rules obtained with fSBI generalized to different network sizes, sparsity, weight initializations, and ratios of E to I, although a few metrics were sometimes outside the predefined ranges (Fig. 5A). We then tested the "plausible" rules on 20-minute-long simulations (versus 2min in Fig. 2&4). Approx. 25% of the polynomial rules still met the conditions on all 15 metrics after 20 minutes (Fig. 5B). Most other rules were disqualified by our ⟨ w IE ⟩ and ⟨ w II ⟩ conditions, which select long-term stable weights, and are thus sensitive to simulation duration. Correspondingly, the II and IE plasticity parameters appeared more refined compared to the starting set (Fig. 5D). Overall, this

A

B

(Xpre)

C

△WEE(0)

asoda

W

Wh1

Wpost

fraction of rules

sampled from

10-1

1

-stable activity

Inh

(A>

Wpre

→

10-2

stableweights

10-3

A.1. dynamics

△WIE(0)!

10-4

=

-plausible

rext

TTo

T1

23

nn1

D

E

π3(0lrexc = 10Hz)

Example plausible rule from T3

rext=5Hz

rext =15Hz

high

Wpre

activity

△WEE

Inh

Wpost

Inh

cov(0i, 0)

0.1

0.1

(ZH)

20

low

0

a

△t (s)

activity

0

0.4

Wee

0.4

1

Wee

high

activity

Wpre

0

△WIE

1.2

1

Wie

1.2

Wie

Wpost

0.8+

0.8

low

55

60

55

60

-0.1

△t (s)

0.1

activity

n

Wpre

Wpost

nl

Wpre

Wpost

-1

t (s)

t (s)

A : Recurrent spiking network receiving Poisson inputs at a random rate r ext, with recurrent EE and IE plastic connections parameterized as multi-layer-perceptrons (MLP). B : Weight changes via a forward pass through an MLP that depends on pre- and post-synaptic traces, current synaptic weight, smoothed membrane potential and two codependency terms at every spike (Methods. 2.2). C : Fraction of rules sampled from each intermediate fSBI posterior that obey the corresponding network metric condition. D : Left: Example plausible candidate rule simulated in two networks with different input rates. Right: pre-post pairing protocol on the EE and IE rules. Note that these rules have different outcomes on the pre-post protocol depending on the network activity (Supp.6.4). E : Pearson correlation matrix of the plasticity rules sampled from the final fSBI posterior, conditioned on the excitatory firing rate being 10Hz.

suggested that the 2min task had properly constrained the EE and EI rules, but was too short to fully constrain the II and IE rules. Similarly, 35% of the 2min-plausible MLP rules met all conditions in the 20min task. Interestingly, dimensionality reduction of the successful plasticity parameters did not reveal sub-structures, suggesting that the 20min task did not dramatically refine the set of filtered rules (Fig. 5C).

4 Discussion and Outlook

Plasticity mechanisms are arduous to probe in vivo and theoretically challenging to describe due to the complex interplay between neural activity and weight dynamics. Here, we introduce fSBI, a tool to numerically infer families of plasticity rules while enforcing strong plausibility constraints on SNN dynamics. We show that fSBI can discover generally plausible plasticity rules in large SNNs, with rules parameterized either as polynomials or MLPs.

fSBI has several advantages compared to previous approaches: analytical methods require various and sometimes strong assumptions; local optimization methods only provide single solutions; and brute-force approaches do not scale (in this study, randomly drawing from the full space of plasticity rules would have required simulating ∼ 100M rules, compared to 200k, Fig. 2C, 4C). However, these approaches are not mutually exclusive: analytical frameworks allow us to understand the sets of rules inferred by fSBI and produce meaningful experimental predictions. Furthermore, the fSBI distributions of plausible rules are a promising starting point for further investigation into specific network computations, potentially suited to local optimization approaches.

In contrast with previous in silico studies of plasticity that produce single predictions, fSBI is a systematic approach to make predictions based on many (if not all) candidate plasticity rules within a search space. For instance, we showed how an isolated EE rule may suggest a Hebbian mechanism, but the fSBI-discovered set contains equally valid anti-Hebbian rules that can establish near identical

A

Polynomial rules

MLPrules

fraction of rules

186420

fraction of rules

186420

-Sparsity x0.5

respecting

日

目

三

respecting

=

三

一

三

=

一

一

- E:l ratio 5:1

Winit X0.5

rexc

u

activity

Wcreep

M

(M)

(3IM)

(/M)

weights

(ISI)A

=

x

u

activity

Wcreep

(M

(3IM)

weights

CV(ISI)

fraction of rules

B

respecting

186429

二

三

－ Polynomial

二

二

二

MLP

二

二

PCA2

C

2.5

t-SNE2

50 ↓

yes

stable

after20min

0.0

0

二

-50

r

activity

M

Wcreep

M

(M)

(3IM)

(M)

weights

CV(ISI)

=

0.0

2.5

T

-50

0

50

no

PCA1

t-SNE1

D

2

1

2

2

2

yes

after20min

stable

8

0

-2

Fo

2

-2

o

αEI

2

-2

-2

-2

α1I

2

no

XEE

or w init EE = w init EI = IE 10 = II 10 = 0 . 15 ). All simulations had a random input strength and initial connectivity. "Activity", "weights" and "A.I" denote the intersection of all metrics in that category. B 10k "plausible" polynomial and 10k MLP rules were simulated on a 20-minute extended version of the 2-minute background task (Fig.2&4). Fraction of the rules that fulfill some plausibility conditions on the 20-minute task. C : First two features of PCA (left), t-SNE (right) applied to the MLP rules in B. Rules fulfilling all plausibility conditions on the 20-minute task are in yellow. D : Non-Hebbian parameters of the polynomial rules from B, with the same color code as C.

network activity (Figs. 2H, 3E), suggesting that studies of single rules or individual connection types may be grossly under-constrained.

More generally, our study challenges the widely used pre-post pairing protocols [1, 6, 44], when used in isolation. Indeed, rules that look near identical under such pre-post protocols may have widely different network-level effects (Fig. 3D, middle and bottom), validating, in hindsight, the need for finely-tuned orchestration in previous studies [8]. Conversely, rules that appear different under observation with a pre-post protocol (Fig. 2H and Fig. 3E) may lead to similar network dynamics. In addition, the results above were obtained in noise-free in silico experiments, suggesting that the connection between empirical data and plasticity rules may be even more complex. The interpretability of the pre-post pairing protocols is further compromised in the MLP-based rules, since these rules include factors linked to plasticity that are not explicitly constrained by such lowdimensional protocols, leading to widely different outcomes depending on the synaptic and neuronal state (Fig. 4D). Additionally, we have not touched on the effects of neuromodulation in our study, something that is currently out of scope but will become the focus of future studies.

Our approach naturally comes with limitations and idiosyncrasies. First, we cannot prove the longterm stability of the rules, as we are limited by the numerical approach and associated demanding compute requirements. On longer timescales, slower homeostatic mechanisms may play a part in synaptic dynamics [45, 46]. Another potential limitation of fSBI is the curse of dimensionality, inherent to most SBI approaches [36], although some recent improvements have been proposed [47-50]. Finally, further analysis is needed to forge robust and meaningful predictions from the commonalities of the large set of inferred rules.

In summary, we meta-learn plasticity rules with an inference-based method, fSBI. This method can be applied to large spiking networks with flexibly parameterized plasticity rules and narrow down the set of promising plasticity mechanisms.

5 Acknowledgments

We thank Chaitanya Chintaluri, Everton Agnes, Nicoleta Condruz, Douglas Feitosa Tomé, Michael Deistler and Jan Boelts for helpful discussions and feedback on the manuscript. This work was funded by the European Research Council (ERC consolidator grant SYNAPSEEK), the German Research Foundation (DFG; Germany's Excellence Strategy MLCoE - EXC number 2064/1 PN 390727645), the German Federal Ministry of Education and Research (BMBF; Tübingen AI Center, FKZ: 01IS18039A), the Human Frontier in Science Program (RGY0076/2018) and the FENS-Kavli Network of Excellence scientific exchange program. This research was supported by the Scientific Service Units (SSU) of IST Austria through resources provided by Scientific Computing (SciComp).

References

Tim VP Bliss and Terje Lømo. Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. The Journal of physiology , 232(2):331-356, 1973.

Larry F Abbott and Sacha B Nelson. Synaptic plasticity: taming the beast. Nature neuroscience , 3(11):1178-1183, 2000.

Jeffrey C Magee and Christine Grienberger. Synaptic plasticity forms and functions. Annual review of neuroscience , 43:95-117, 2020.

Ami Citri and Robert C. Malenka. Synaptic plasticity: Multiple forms, functions, and mechanisms. Neuropsychopharmacology , 33:18-41, 2008.

Amanda R. McFarlan, Christina Y. C. Chou, Airi Watanabe, Hannah Owens Nicole Cherepacha, Maria Haddad, and P. Jesper Sjöström. The plasticitome of cortical interneurons. Nature Reviews Neuroscience , 24, 2022.

Wulfram Gerstner, Richard Kempter, J Leo Van Hemmen, and Hermann Wagner. A neuronal learning rule for sub-millisecond temporal coding. Nature , 383(6595):76-78, 1996.

Tim P Vogels, Henning Sprekeler, Friedemann Zenke, Claudia Clopath, and Wulfram Gerstner. Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science , 334(6062):1569-1573, 2011.

Friedemann Zenke, Everton J Agnes, and Wulfram Gerstner. Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nature communications , 6(1):1-13, 2015.

Ashok Litwin-Kumar and Brent Doiron. Formation and maintenance of neuronal assemblies through synaptic plasticity. Nature Communications , 5, 2014.

Everton J Agnes and Tim P Vogels. Codependent excitatory and inhibitory plasticity accounts for quick, stable and long-lasting memories in biological networks. BioRxiv , page 437962, 2022.

Claudia Clopath, Lars Büsing, Eleni Vasilaki, and Wulfram Gerstner. Connectivity reflects coding: a model of voltage-based stdp with homeostasis. Nature Neuroscience , 13:344-352, 2010.

Sukbin Lim, Jillian L. McKee, Yali Woloszyn, Luke Amit, David J. Freedman, David L. Sheinberg, and Nicolas Brunel. Inferring learning rules from distribution of firing rates in cortical neurons. Nature neuroscience , 18:1804-1810, 2015.

Yoshua Bengio, Samy Bengio, and Jocelyn Cloutier. Learning a synaptic learning rule . Citeseer, 1990.

Keren Gu, Sam Greydanus, Luke Metz, Niru Maheswaranathan, and Jascha Sohl-Dickstein. Meta-learning biologically plausible semi-supervised update rules. bioRxiv , 2019.

Basile Confavreux, Friedemann Zenke, Everton J Agnes, Timothy Lillicrap, and Tim P Vogels. A meta-learning approach to (re) discover plasticity rules that carve a desired function into a neural network. Advances in Neural Information Processing Systems 34 (NeurIPS) , 2020.

Luke Metz, Niru Maheswaranathan, Brian Cheung, and Jascha Sohl-Dickstein. Learning unsupervised learning rules. arXiv preprint arXiv:1804.00222 , 2018.

Jakob Jordan, Maximilian Schmidt, Walter Senn, and Mihai A Petrovici. Evolving interpretable plasticity for spiking networks. eLife , 10:e66273, 2021.

Jack Lindsey and Ashok Litwin-Kumar. Learning to learn with feedback and local plasticity. Advances in Neural Information Processing Systems 34 (NeurIPS) , 2020.

Danil Tyulmankov, Guangyu Robert Yang, and LF Abbott. Meta-learning synaptic plasticity and memory addressing for continual familiarity detection. Neuron , 110(3):544-557, 2022.

Navid Shervani-Tabar and Robert Rosenbaum. Meta-learning biologically plausible plasticity rules with random feedback pathways. Nature Communications , 14(1):1805, 2023.

Wulfram Gerstner and Werner M Kistler. Mathematical formulations of hebbian learning. Biological cybernetics , 87(5):404-415, 2002.

Jean-Pascal Pfister and Wulfram Gerstner. Triplets of spikes in a model of spike timingdependent plasticity. Journal of Neuroscience , 26(38):9673-9682, 2006.

Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition . Cambridge University Press, 2014.

Peter Dayan and Laurence F Abbott. Theoretical neuroscience: computational and mathematical modeling of neural systems . MIT press, 2005.

Nicolas Brunel. Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. Journal of computational neuroscience , 8:183-208, 2000.

Tim P Vogels, Kanaka Rajan, and Larry F Abbott. Neural network dynamics. Annu. Rev. Neurosci. , 28:357-376, 2005.

Alfonso Renart, Jaime De La Rocha, Peter Bartho, Liad Hollender, Néstor Parga, Alex Reyes, and Kenneth D Harris. The asynchronous state in cortical circuits. science , 327(5965):587-590, 2010.

A. Sanzeni, M. H. Histed, and N. Brunel. Emergence of irregular activity in networks of strongly coupled conductance-based neurons. Physical Review X , 12(1):011044, 2022.

Michael N Shadlen and William T Newsome. Noise, neural codes and cortical organization. Current opinion in neurobiology , 4(4):569-579, 1994.

Zachary F Mainen and Terrence J Sejnowski. Reliability of spike timing in neocortical neurons. Science , 268(5216):1503-1506, 1995.

Sen Song, Per Jesper Sjöström, Markus Reigl, Sacha Nelson, and Dmitri B Chklovskii. Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS biology , 3(3):e68, 2005.

Michael Okun and Ilan Lampl. Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities. Nature neuroscience , 11(5):535-537, 2008.

Jonathan W Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher, Alan M Litke, EJ Chichilnisky, and Eero P Simoncelli. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature , 454(7207):995-999, 2008.

Marlene R Cohen and Adam Kohn. Measuring and interpreting neuronal correlations. Nature neuroscience , 14(7):811-819, 2011.

George Papamakarios and Iain Murray. Fast ϵ -free inference of simulation models with Bayesian conditional density estimation. In Advances in Neural Information Processing Systems 29 , pages 1028-1036. Curran Associates, Inc., 2016.

Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences , 117(48):30055-30062, 2020.

Pedro J Gonçalves, Jan-Matthis Lueckmann, Michael Deistler, Marcel Nonnenmacher, Kaan Öcal, Giacomo Bassetto, Chaitanya Chintaluri, William F Podlaski, Sara A Haddad, Tim P Vogels, et al. Training deep neural density estimators to identify mechanistic models of neural dynamics. Elife , 9:e56261, 2020.

Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. Advances in neural information processing systems , 30, 2017.

David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-free inference. In International Conference on Machine Learning , pages 24042414. PMLR, 2019.

Michael Deistler, Jakob H Macke, and Pedro J Gonçalves. Energy-efficient network activity from disparate circuit parameters. Proceedings of the National Academy of Sciences , 119(44): e2207632119, 2022.

Michael Deistler, Pedro J. Goncalves, and Jakob H. Macke. Truncated proposals for scalable and hassle-free simulation-based inference. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Conor Durkan, George Papamakarios, and Iain Murray. Sequential neural methods for likelihood-free inference. Bayesian Deep Learning Workshop at Neural Information Processing Systems , 2018.

Henry Markram, Joachim Lübke, Michael Frotscher, and Bert Sakmann. Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science , 275(5297):213-215, 1997.

Guo-qiang Bi and Mu-ming Poo. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of neuroscience , 18(24):10464-10472, 1998.

Keith B. Hengen, Alejandro Torrado Pacheco, James N. McGregor, Stephen D. Van Hooser, and Gina G. Turrigiano. Neuronal firing rate homeostasis is inhibited by sleep and promoted by wake. Cell , 165(1):180-191, 2016. ISSN 0092-8674.

Gina G. Turrigiano. The self-tuning neuron: Synaptic scaling of excitatory synapses. Cell , 135 (3):422-435, 2008. ISSN 0092-8674.

Poornima Ramesh, Jan-Matthis Lueckmann, Jan Boelts, Álvaro Tejero-Cantero, David S. Greenberg, Pedro J. Goncalves, and Jakob H. Macke. GATSBI: Generative adversarial training for simulation-based inference. In International Conference on Learning Representations , 2022.

Lorenzo Pacchiardi and Ritabrata Dutta. Likelihood-free inference with generative neural networks via scoring rule minimization. arXiv preprint arXiv:2205.15784 , 2022.

Victor M.-H. Ong, David J. Nott, Minh-Ngoc Tran, Scott A. Sisson, and Christopher C. Drovandi. Likelihood-free inference in high dimensions with synthetic likelihood. Computational Statistics &Data Analysis , 128:271-291, 2018. ISSN 0167-9473.

Athanasios Kousathanas, Christoph Leuenberger, Jonas Helfer, Mathieu Quinodoz, Matthieu Foll, and Daniel Wegmann. Likelihood-free inference in high-dimensional models, 2015.

Friedemann Zenke and Wulfram Gerstner. Limits to high-speed simulations of spiking neural networks using general-purpose computers. Frontiers in neuroinformatics , 8:76, 2014.

Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature , 585(7825):357-362, September 2020.

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011.

Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, ˙ Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods , 17:261-272, 2020.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. CoRR , abs/1912.01703, 2019.

Alvaro Tejero-Cantero, Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pedro J. Gonçalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulationbased inference. Journal of Open Source Software , 5(52):2505, 2020.

J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering , 9 (3):90-95, 2007.

Nikolaus Hansen. The CMA evolution strategy: A tutorial. arXiv preprint , 1604.00772, 2016.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel meta-learning approach based on Bayesian filtering for discovering synaptic plasticity rules that entrain a E-I recurrent spiking neural network to satisfy a set of predefined criteria (e.g. the network firing rates occupy a reasonable regime at steady-state, with appropriate variability statistics, etc.). Using two different parameterizations of their synaptic plasticity rule (a polynomial plasticity rule resembling many different kinds of experimentally observed STDP vs. a multilayer perceptron parameterization) the authors show that they are able to recover many heterogenous rules that match their pre-specified target dynamical regimes, which in some cases align to predictions made by a mean-field model, and in other cases differ in interesting ways.\n\nSTRENGTHS:\nThe paper is well written and very clear. Furthermore, the comparisons between the families of plasticity rules derived from mean-field analytic approximations and those derived from fSBI are very interesting--that the authors found plasticity rules that would not have been predicted to be effective under the mean-field approximation is an endorsement of the value of their method, and the fact that several similar-seeming plasticity rules were found to produce markedly different dynamical spiking regimes  has interesting implications for the interpretation of experimental results. The authors' emphasis on multiple different learning rules for different cell types is a welcome improvement over previous methods, and it is a further benefit of their approach that fSBI produces entire families of plasticity rules rather than requiring full commitment to any one particular learning rule (this may allow certain types of predictions across cell types which I elaborate on below).\n\nWEAKNESSES:\nThough the results are technically solid, thorough, and interesting enough to warrant publication in my opinion, I have serious doubts about the long-term value of this conceptual framework overall for several reasons that I think it would be very helpful for the authors to elaborate on.\n\nFirst of all, the learning rules uncovered by fSIB do not actually do anything beyond stabilize the network in a reasonable dynamical regime. It is unclear whether these forms of plasticity can be used for any other interesting functions, such as memory formation or task performance. While this is not an unattainable goal for meta-learning approaches in general (see e.g. Confavreux & Vogels 2020), these alternative approaches use either gradient-based or in some cases evolutionary learning strategies which are likely better suited to searching in high dimensional spaces than sampling-based methods such as fSIB. Without learning to perform ecologically relevant tasks and without comparing to existing alternative methods, we are left wondering whether fSIB is actually useful as an optimization method in this space.\n\nSecond, and relatedly, the flexibility of the plasticity rules learned by fSIB (and most meta-learning alternatives) is limited by the imaginations of the algorithm designers & the scalability of the algorithm. For instance, there are a wide variety of plasticity algorithms that have been proposed for non-trivial task learning that would not be implementable within the parametric space proposed by the authors (e.g. reward-modulated plasticity, as the authors mention; burst-mediated learning as in Payeur et al. 2021; any form of learning involving dendritic compartmentalization as in Urbanczik & Senn 2014; or possibly even forms of phenomenological plasticity based on spiking triplets etc.). The search space is so wide, that all one may do is select a space that accommodates some known set of existing plasticity rules, which makes one wonder whether meta-learning methods for plasticity are in general doing little more than rediscovering algorithms that were already previously identified, but without the theoretical guarantees provided by algorithms motivated by optimization principles.\n\nThis second point would be less relevant if it were possible to generate genuine, testable predictions based on meta-learned plasticity algorithms that go above and beyond those already identified in existing literature. In this study, there appear to be some plasticity rules identified that are not already supported by the mean-field model. However, we are provided with a wide ensemble of potential plasticity parameterizations, and it's unclear what to actually do with this distribution. For instance, could it be used to infer, given a plasticity rule for one type of neuron (e.g. EE synapses), a probability distribution over the parameters of the other synapses (EI, IE, II)? Should we expect more variability in plasticity parameters in biology in dimensions of parameter space that are not constrained by the mean-field equations for stabilizing the network?\n\nQUESTIONS:\nThe scaling of the algorithm doesn't seem to be analyzed much beyond one parameterization each for the polynomial and MLP cases which work effectively. Does the MLP method scale well as the number of optimized parameters increases?\n\nHow well do the learned plasticity rules generalize to any of the following: to different network architectures (e.g. multiple layers or different connectivity probabilities), to changes in neuron properties (e.g. excitability or spiking properties), or variations in input statistics?\n\nLIMITATIONS:\nThere are no obvious potential negative societal impacts of the proposed work, and the authors do address many of the limitations of their work (I mention a few that they do not address above).", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present the filter Simulation-Based Inference (fSBI) method, aimed at automating the discovery of families of rules that establish plausible dynamics within large spiking networks. This study seeks to overcome the challenges presented by human intuition and the overly simplified assumptions that have characterized prior research in synaptic plasticity—a critical element in brain-based learning and memory processes. The fSBI method facilitates the extensive exploration of the potential rule space, identifying sets of rules via metrics that effectively constrain network dynamics to satisfy the intended conditions. Importantly, this approach evaluates one metric at a time, significantly reducing computational complexity compared to simultaneous evaluation of multiple metrics.\n\nOverall, this study provides a principled approach for meta-learning a family of plasticity rules in networks, offering a starting point for deeper analysis of computation and plasticity mechanisms. Additionally, the study highlights that classical experimental predictions often yield contradictory results when considered in isolation, thereby challenging traditional theoretical and experimental protocols for understanding plasticity. Furthermore, the inherent flexibility of this tool lends it the potential to incorporate new neuroscience findings into the metrics and architecture as emerging data flows in.\n\nSTRENGTHS:\no\tThe manuscript is very well-written, with informative illustrations.\n\no\tThe proposed fSBI framework exhibits flexibility across architectures and rule parameterization, and can be applied in unsupervised learning contexts.\n\no\tThe viability of the learned rules and their resulting network-level effects is theoretically grounded based on mean-field predictions.\n\no\tAnalysis pertaining to the reduction in effective dimensionality of rule parameters and interdependence of plasticity rule mechanisms is insightful.\n\nWEAKNESSES:\no\tDespite only evaluating one metric at a time, the method may still prove computationally demanding if the number of rule parameters is significant, to accommodate a broader rule basis. The authors recognize this \"curse of dimensionality\", and the Appendix notes simulations took two months.\n\no\tA minor point, but the paper could benefit from more detailed explanation on how specific values for the metric bounds were chosen and how sensitive these choices affect the results (in addition to what's already in 5.2).\n\no\tI could not find information on whether simulations were repeated across different initializations and seeds.\n\nQUESTIONS:\no\tAlthough the study acknowledges that slower homeostatic mechanisms aren't considered, how might the inclusion of weight regularization (or homeostatic mechanisms) influence the pool of plausible rules and the interaction between plasticity terms? \n\no\tNeuromodulation is not accounted for in this study, but could the authors briefly discuss (perhaps in Discussion) the feasibility of extending this method to infer rule parameters in recently proposed modulated-eligibility trace-based biologically plausible learning rules for training recurrent spiking neural networks [1-3]?\n\no\tCould the authors provide their intuition/hypotheses on potential reasons why rules appearing similar/different under pre-post protocols can result in different/similar network-level effects?\n\no\tWhat does the resulting weight distribution look like for the plausible candidates? Is it lognormal?\n\no\tReferences: \n\n\t[1] Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11(1):1–15, dec 2020.\n\n\t[2] James M Murray. Local online learning in recurrent networks with random feedback. ELife,8:e43299, 2019.\n\n\t[3] Yuhan Helena Liu, Stephen Smith, Stefan Mihalas, Eric Shea-Brown, and Uygar Sümbül. Cell-type–specific neuromodulation guides synaptic credit assignment in a spiking neural network. Proceedings of the National Academy of Sciences, 118(51), 2021.\n\nLIMITATIONS:\nThe authors have acknowledged various limitations: the curse of dimensionality associated with fSBI, limited interpretability of MLP, the use of noise-free simulations, not considering long-term stability measures, among others. These limitations might impact the study's conclusions, including the pool of plausible rules and the nature of interactions between plasticity terms. However, despite these constraints, the study introduces an encouraging framework (fSBI) for inferring plasticity rules that could be enhanced in future research to incorporate additional considerations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript introduces a novel way of sampling plasticity rules that obey certain plausibility constraints. Specifically, the authors focus on networks of excitatory and inhibitory spiking neurons and use a simulation-based inference (SBI) technique to filter plasticity rules based on the behavior of the networks in which they are implemented. They demonstrate the effectiveness of the technique, and in particular find that standard neuroscience protocols for examining synaptic plasticity are insufficient for identifying plasticity rules.\n\nSTRENGTHS:\n*Originality:*\nThis is a nice extension of simulation-based inference that significantly reduces the size of the search space by enforcing constraints one by one instead of all at the same time.\n\n*Quality:*\nThe work seems carefully done, with a significant number of simulations to support their conclusions.\n\n*Clarity:*\nThe presentation is clear and the paper is well-written.\n\n*Significance:*\nUnderstanding the plasticity rules used by neural circuits is of utmost importance in neuroscience. This paper addresses this question in an interesting way, and the authors show shortcomings with standard neuroscientific approaches.\n\nWEAKNESSES:\nThere are a number of choices related to the simulations that seem insufficiently discussed: e.g., the ratio of excitatory to inhibitory neurons in the networks that were simulated; the sizes of these networks; the statistics of the inputs to the networks; the initial weight distribution; the duration of the simulations; the various numerical constants used in the metrics from section 2.4.\n\nAlso, while the authors point out that standard protocols from neuroscience do not uniquely pin down bio-plausible plasticity rules, it would be good to at least speculate on possible approaches that could solve this problem.\n\nQUESTIONS:\n1. Do we need to worry about any biases in the sampling obtained using fSBI?  If there are such biases, would they depend markedly on details such as the order in which the constraints are imposed?\n2. How were the details of the simulations mentioned above (like size of networks) chosen? Were multiple choices explored and the choice did not matter? Were some of the choices informed by measurements in real brains? If so, which areas and which animals -- and can we expect the results to generalize?\n3. On line 82 it is written that *\"For the MLP plasticity rule, only EE and IE synapses undergo spike-triggered updates\"*. Why? Is this an arbitrary choice, or is there something deeper about it?\n4. The comment on lines 268–270 suggests that many of the MLP rules are not, in fact, realistic. Should we then not try to either simplify the neural nets or add more constraints to restrict ourselves to realistic rules?\n\nMinor comments:\n* on line 86, what are the superscripts (1), (2)? are they the same as *pre*, *post*?\n* on line 195, *\"the order of the least to the most restrictive metric\"*: how did you know which metrics were more restrictive?\n* Figure 3, panel A: what do the colors for the axis labels represent?\n* Figure 3, panels D, E: which side of the color bar is for small distances and which is for big?\n* on lines 239-240: *\"fSBI rules lead to plausible network dynamics both when simulated with only EE parameters [...] and also when using the rule for all connection types\"*: why would we expect / want the dynamics to be stable when the plasticity rule applies only to some of the connections in the network?\n* line 257: should mention the activation function(s) used for the MLP\n\nLIMITATIONS:\nThe authors have adequately discussed limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nHow neurons update their synaptic strengths is an important and active area of research in neuroscience, with potential applications in ML. Much work has been done using analytically tractable mean-field models, in large part because the search space for possible rules is intractably large.  Here, the authors introduce an approach that cuts the search space down significantly.  While simulation-based inference has been used to infer parameter distributions that lead to network dynamics, the authors add constraints on plausible network dynamics to further reduce the search space to a manageable size.  \n\nThey find that plasticity rules predicted by the mean field model they derive do not always result in plausible dynamics in the full network.  Moreover, pre-post diagrams that are often used in STDP analyses were found to be of limited use: rules with very similar diagrams could drastically diverge in terms of network dynamics, while rules with very similar network dynamics could result in very different pre-post diagrams.\n\nSTRENGTHS:\nOverall, this is a good study that contributes to the field. Brute force strategies are inherently problematic; the authors' filtering approach provides a helpful middle ground between approaches based on more tractable methods and full brute-force grid searches.  The comparison of the rules between the full and mean-field networks, as well as the findings of the pre-post diagrams, add nuance to the study of plasticity in computational and experimental settings.\n\nWEAKNESSES:\nMajor:\n\nThe filter constraints are hard boundaries that are not always obviously well-founded.  For example, it isn't clear why firing rates above 50 Hz are considered implausible when population gamma oscillations typically go up to ~80 Hz and neurons have been found to fire up to ~1000 Hz. Why the bounds for $<w_{EX}>$ and $<w_{IX}>$ are an order of magnitude apart is unclear, nor is there physiological comparison (weights are in arbitrary units). Other bound decisions are similarly left ambiguous.  \n\nIf the authors believe there are good reasons to have chosen the specific values presented, they should explain and cite the relevant work.  Otherwise, the paper should be rewritten to soften the claims.  In particular, the authors could instead choose to indicate that others might choose somewhat different bounds to be more or less inclusive, but that this is a first attempt and that the main conclusions are left unaffected.  The \"plausible\" network dynamics claims should thus be de-emphasized in that case.\n\nMinor: \n\nThe MLP plasticity rule was introduced in a manner that was difficult to understand.  However, I found my questions answered when the results were introduced.  In particular, lines 252-264 should either be moved to line 82, when the rule is introduced (or integrated into that section), or vice versa.\n\nThe color bars in Fig 3D-E should have scales; if the lower bound is not very near zero, some of the associated claims may need to be softened.\n\nLine 84: $C_i^I$ should be $C_i^{\\mbox{inh}}$.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-28"}