Discovering plasticity rules that organize and maintain neural circuits
Authors: David G Bell
Alison Duffy
Adrienne Fairhall
Venue: NeurIPS 2024 poster (2024)
Keywords: biologically plausible learning rules plasticity self-organization RNNs homeostasis meta-learning
Primary area: neuroscience_and_cognitive_science
TL;DR: A supervised approach yields biologically plausible learning rules that self-organize and maintain robust representations of time within RNNs.

Abstract: Intrinsic dynamics within the brain can accelerate learning by providing a prior scaffolding for dynamics aligned with task objectives. Such intrinsic dynamics would ideally self-organize and self-sustain in the face of biological noise including synaptic turnover and cell death. An example of such dynamics is the formation of sequences, a ubiquitous motif in neural activity. The sequence-generating circuit in zebra finch HVC provides a reliable timing scaffold for motor output in song and demonstrates a remarkable capacity for unsupervised recovery following perturbation. Inspired by HVC, we seek a local plasticity rule capable of organizing and maintaining sequence-generating dynamics despite continual network perturbations. We adopt a meta-learning approach introduced by Confavreux et al, which parameterizes a learning rule using basis functions constructed from pre- and postsynaptic activity and synapse size, with tunable time constants. Candidate rules are simulated within initially random networks, and their fitness is evaluated according to a loss function that measures the fidelity with which the resulting dynamics encode time. We use this approach to introduce biological noise, forcing meta-learning to find robust solutions. We first show that, in the absence of perturbations, meta-learning identifies a temporally asymmetric generalization of Oja's rule that reliably organizes sparse sequential activity. When synaptic turnover is introduced, the learned rule incorporates a form of homeostasis, better maintaining robust sequential dynamics relative to other previously proposed rules. Additionally, inspired by recent findings demonstrating that the strength of projections from inhibitory interneurons in HVC also dynamically responds to perturbations, we explore the role of inhibitory plasticity in sequence-generating circuits. We find that learned plasticity adjusts both excitation and inhibition in response to manipulations, outperforming rules applied only to excitatory connections. We demonstrate how plasticity acting on both excitatory and inhibitory synapses can better shape excitatory cell dynamics to scaffold timing representations.

Discovering plasticity rules that organize and maintain neural circuits
David Bell Department of Physics University of Washington Seattle, WA 98195
Alison Duffy
Department of Physiology and Biophysics University of Washington
Seattle, WA 98195
davidgbe@uw.en
Adrienne Fairhall
Department of Physiology and Biophysics University Washington
Seattle, WA 98195
Abstract
Intrinsic dynamics within the brain can accelerate learning by providing a prior scaffolding for dynamics aligned with task objectives. Such intrinsic dynamics should self-organize and self-sustain in the face of fluctuating inputs and biological noise, including synaptic turnover and cell death. An example of such dynamics is the formation of sequences, a ubiquitous motif in neural activity. The sequence- generating circuit in zebra finch HVC provides reliable timing scaffold for motor output in song and demonstrates remarkable capacity for unsupervised recovery following perturbation. Inspired by HVC, we seek local plasticity rule capable of organizing and maintaining sequence-generating dynamics despite continual network perturbations. We adopt meta-learning approach introduced by Confavreux et al, which parameterizes learning rule using basis functions constructed from pre- and postsynaptic activity and synapse size, with tunable time constants. Candidate rules are simulated within initially random networks, and their fitness evaluated according to loss function that measures the fidelity with which the resulting dynamics encode time. We use this approach to introduce biological noise, forcing meta-learning to find robust solutions. We first show that, in the absence of perturbation, meta-learning identifies a temporally asymmetric generalization of Oja's 's rule that reliably organizes sparse sequential activity. When synaptic turnover is introduced, the learned rule incorporates additional form of homeostasis, better maintaining sequential dynamics relative to other previously proposed rules. Additionally, inspired by recent findings demonstrating plasticity in synapses from inhibitory interneurons HVC, we explore the role of inhibitory plasticity in sequence generaaing circuits. We find that learned plasticity adjusts both excitation and inhibition in response to manipulations, outperforming rules applied only excitatory connections. We demonstrate how plasticity acting both excitatory and inhibitory synapses can better shape excitatory cell dynamics scaffold timing representations.
Introduction and related work
How computational structures are organized and maintained within the brain is a central question within neuroscience. While feedback is clearly essential for learning, self-organization of neural circuits can unfold without feedback, e.g. during development. Brains have evolved specific cell types with nonrandom spatial organization, plasticity rules, and connectivity that likely introduce
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

b
Inhibitory
Neural representation time
Output(t)
Call index
Feedback(t)
Excitatory neurons
Time (ms)
Decoder trained
Evaluate loss
Σ
ca (x,, wby
Decoder error Activity penalty + Synaptic penalty
L(c, T) =
h
Decode time from neural activity
C, T
Network activation
(c, T)
Synaptic turnover
Simulate candidate plasticity rules
Adjust rules genetic algorithm to minimize loss
Figure I: Meta-learning approach to discovering plasticity rules that organize sequences (a) In zebra finch song learning, neural representation of time (left) in HVC simplifies the sequential motor learning task of producing the correct spectral output. (b) Putative network structure of zebra finch HVC: feed-forwane forward, excitatory network with recurrent inhibition (left). HVC excitatory neurons (red) fire sparsely in time while interneurons (blue) fire tonically (right). (c) Strategy for learning plasticity underlying sequence organization: candidate plasticity rules, parameterized by set of coefficients and time constants, are simulated. A loss function is evaluated on the resulting dynamics, and new candidate rules generated. (d) Test procedure for representation of time. Networks activated 400 times (red bars). From the final 50 activations, six are chosen to train a decoder and six to test the representation by decoding time from neural activity. (e) Discovery of robust plasticity rules is encouraged by introducing synaptic turnover, the stochastic addition and removal of synapses, into simulations.
strong set of inductive biases the information processing they perform. How might organization of useful circuit dynamics be established and maintained throughout life without the need for feedback? Recent work suggests self-organized computations, once established, can accelerate learning and improve performance when experience is limited: Nicola and Clopath [T] demonstrated that stable high dimensional time signal could improve a network's 's performance on sequential motor tasks (Fig. [h). Najarro and Risi [2] learned Hebbian plasticity that orchestrated spontancous walking behavior quadruped agents; similar work has shown architectural priors increase the sample efficiency and generalization of RL approaches to locomotion [3 4). Additionally, in RL settings, supplying agents with time input permits them to adopt demedeeendeet policies (5). The ability of computational primitives, such as timing representations, to self-organize is challenged by the shifting structure of neural circuits. Synaptic loss, synaptogenesis, cell death, and neurogenesis pose challenges for all learning algorithms, but particularly for self-organizatioon which must be based solely on local information rather than global task performance.
Here, we aim to find plasticity rules that self-organize and maintain one useful computational primitive: sparse, sequential activity. Such activity is widely seen in many areas of the brain including hippocampus (6). cortex [7]. and basal ganglia [8] In the songbird zebra finch, area HVC (used as proper noun), cortical-lic region, displays sequential activity representing time 9 reducing the problem of motor learning to driving the correct motor neuron at the correct moment [10]. Extensive literature has explored how such sequence- sequence-generating circuits could emerge in the absence of feedback I7 but has largely focused on either how these structures organize or how they self-maintain, using guessed plasticity rules, and neglecting the effects of ongoing synaptic noise. Further, previous work on sequence organization within HVC has focused on plasticity between excitatory (E) neurons. Recent experimental findings show unsupervised recovery of HVC dynamics is accompanied by changes in both E-EE and also nabiitory-to-excitaan (I++) synaptic strength [18).

---

Here, instead of imposing a guessed rule, we ask which self-supervised plasticity rules can organize and maintain sequential dynamics within a network. We employ meta-learning, a supervised method to learn learning rules [3), in order to discover rules that self-organize sequence. Our approach stems from a rich history of learning local plasticity rules, including rules that extract representations
from data (24), enhance artificial agent performance on familiarity and navigation tasks [2] 25], and explore
seaggiccall-plassbblee replacements or complements to backpropagation [19 20 26].
In this study, we parameterize the space of plasticity rules with a basis of activity- and synapse size-dependent terms. The set of coefficients weighting these terms and associated time constants adjusted to minimize loss function. Inspired by the HVC context, we pose the loss in terms of the accuracy with which the time since an initial network input can be decoded from the circuit dynamics. We first consider E-E rules alone, and then add I->E and E-xI plasticity. We then introduce perturbations to the circuit and investigate which learned plasticity rules promote circuit stability. We find that meta-learned rules for self-organized sequence generation and maintenance contain distinct forms of spike-timing dependent plasticity, homeostasis, and network bounds, which outperform previously proposed sequence-organizing rules in the presence of noise and that plasticity on reciprocal connectivity to inhibitory neurons confers additional stability on the network dynamics. Our main contribution the exploration of unsupervised and unrewarded plasticity via meta-learning that organizes and maintains a specific and biologically relevant computational motif, a sequence.
1.1 Background on zebra finch physiology
In the zebra finch, nucleus HVC contains excitatory neurons that fire sparsely (typically in one burst of spikes) during song and are purportedly arranged in a feed-forward structure (Fig. [T) [27]. A subset of these cells, known (VC(RA) neurons, project to downstream nucleus RA (robust nucleus of the archistriatum), which in turn projects to vocal neurons of the syrinx and to the brainstem, which regulates respiration (9. HVC receives excitatory projections from nucleus Uva, which controls
the onset of song syllables (8 and provides input for the duration of song (29] HVC neurons
inhibit each other disynaptically via a population of inhibitory interneurons (3) Remarkably, singing
behavior can persist when the nucleus is transected (31, demonstrating its resilience.
2 Results
2.1 Learning biologically plausible plasticity on E-+E synapses that organizes sequence-generating circuit
We first learned a plasticity rule on E-EE synapses that organizes randomly connected network into a sequence-generatinn circuit in the absence of any perturbation. We initially constrained plasticity to E-EE synapses to compare with previously proposed rules, which have largely only considered E-EE plasticity. We did not include rules that imposed hard bounds on the size of individual synapses or on the collective strength of all synapses onto neuron, as we aimed learn plasticity rules that could permit flexible rescaling of connections in response to perturbations, as has been observed experimentally 18]
We use a network of 25 E and 8 I threshold-linea. neurons. Each neuron fired according to zt)))
(V) -|+ where (t) evolves via Tm,V (t) Vt) + Σ, mt)ar((). Here, Way is the weight of the synapse + j. Tm is the membrane time constant, and is the bias. Initial connectivity (Fig. ( was random, but contained no I->I connectivity as is the case in HVC [30 32 (see Supp. Sec. 1 for all network model details).
2.1.1 Meta-learning procedure
We adopt a meta-learning approach pioneered by Bengio et al. [19 and extended by Confavreux et al 33 We parameterize a set of plasticity rules with coefficients e₁ and time constants T&e such that individual synapses, Wij. evolve according to
w = (()))(()) Σ co Fo (z))))
(1) TR)
where and xy are pre- and postsynaptic activities, respectively, θ is the Heaviside function, and FR is the th term in the plasticity rule (see Fig. 3 Supp. Sec. 2 for all terms). The basis includes
3

---

b
10"
Rule coefiiient
2.5
5000
R2
Loss
-2.5
2000
2000
Cellindex
Epochs
Epochs
d
25
Cell index
2
0
25
-
Cell
Weight
50
50
50
Time (s)
Figure 2: Meta-learning discovers unsupervised local plasticity rules that organize sequential activity (a) Evolution of coefficients and time constants during meta-learning. (b) Training loss (black) and test loss (blue) during meta-leamiing Inset: median decoding accuracy for 14 learned rules across 100 networks (blue points) compared to no plasticity (dashed gray line). (c) Meta-learned plasticity rules generate sequence dynamics. Network activity during the first activation (left), 200 (middle), and 400*h (right), sorted by ordering of mean firing time of the final activation (425th). Note: plasticity rule is held fixed during simulation. (d) Weight matrices at activation I and 400, sorted based on mean firing time of the final activation. Connectivity between E cells organizes into a feedforward structure.
terms that filter pre- and postsynaptic activities with decaying exponentials (denoted e.g. 2,) as these terms convey information about the durations of activations and relative ordering, not just their instantaneous rates. Synapses evolving under Eq. were bounded so that they obeyed Dale's 's law.
We next define a loss function that evaluates the quality of the sequential dynamics organized under a chosen {Ck, } and attempt to minimize it using an evolutionary strategy, Covariance Matrix Adaptation (CMA-ES) 34 We use CMA-ES to sample from the space of possible {ck, ти and evaluate the loss at each point by simulating 10 randomly initialized networks under the given rule and evaluating the resultant dynamics the end of the simulation. Each simulation is divided into 400 activations of 110 ms. At 10 ms of each activation, single fixed neuron is driven by a strong kick of excitation [17. Following this, all other neurons in the network receive Poisson distributed input for a period of 65 ms. A fraction of this Poisson input is held fixed from trial to trial, mirroring input to HVC from the nucleus Uva, which likely does not provide a fully stochastic signal to the downstream area, 28 The total loss for given rule is simply the sum of losses across each of the 10 networks.
2.1.2 Loss function
To learn sequences, we define a loss function based on three principles: (1) elapsed time since initial network activation should readily decoded from network activity, (2) total network activity should be sparse, and (3) total synaptic change in the network should be minimized. The latter two principles impose the assumption that effective plasticity does not require excess network activity or synaptic change to stabilize network function. Our loss is
1 L(C,T) Ld, T) Ao Σ|" xz(()tt++ (PP(eT))
(2)
where
((c,t)=E (a.Faat)) 20), (t)(t) Tx) ((1u, (t)ldt. 1.j.h
(3)
P(c, T) penalizes the all synaptic changes due to each component of F. This Likei penalty on penalizes each term not by the size of the term's coefficient, C₁ but by the quantity of synaptic change it evokes. While penalizing the magnitude of is standard (0 33 we take this approach to compare different terms on common scale, as each component of F has differing dependence on 2; Iy, and Way. Xa and 0g positive constants weighting the activity and synaptic change penalties.

---

a
Coefficient
b
109
-0.02
0.02
10
Const
Loo
250
Vin
Yos
Kie
None
Const
*
xy
You
Kar
Xon
You
W9
wxy
wynn
wxy
WAin
WXet
wyst
WXix
Dropped term
C
Const
d
Wynt
109
wx You
KE
W
>300
10e
WXxx
wyeey
10a
WXex
Kine Y
200
Yine y
500
Tay
X
X
Loss
IXe:
Tyecy
100
IXee
W
Wyin
Wx Wxy in
T WKar
WKing
2
3
4
5
r WXXe
WXint
r Wye y
Wynn
r WX
WXint
No. terms included
Duration
Trial
10
Figure 3: Perturbing learned plasticity rules reveals dependence on temporally asymmetric Hebbian learning and postsynaptic activity bound (a) Distributions of coefficients (blue) and time constants (purple) of basis terms across N=14 training instances (individual instances in black). The basis set consists of functions of pre and post-synaptic neural activity and synaptic weights. activity of pre-synaptic neuron; = activity of post-synaptic neuron; w = weight of synapse; Tint (æ: filtered with r). (b) Test loss when individual coefficients are set to zero. (c) Difference in median loss between full Iearned solutions run on 100 test networks and solutions with one term dropped. Separate column denotes median loss for terms across all trials. (d) Progressive refitting of model in order of impact on median loss with and without term. Symbols indicate the term added at each refitting. Losses for 100 test networks shown (blue); medians shown black crosses.
To determine the decoder loss (c T), the activity of networks was sampled at 500 time points of six activations of the network. From these, a linear decoder was constructed and used decode the activity at 200 time points of six subsequent activations. Note that since the initial kick of excitation was time-locked to beginning of each activation, constructing a decoder read time elapsed since the beginning of the activation was equivalent to decoding time elapsed since the initial excitatory kick was presented.
2.1.3 Learned E-> plasticity induces sequences using temporally asymmetric Hebbian learning and a peak postsynaptic activity bound
Meta-learnin reliably found plasticity rules that organized random E synaptic connectivity into feed-forward structures that generated sequences when activated (Fig. --e). The organized structures were not grouped into links, as in a synfire chain, but were better described by a kernel in which the strength of synapse between two cells depended the lag between their mean firing times (Fig. 21, right). The set of rules discovered by meta-learning are not sparse in the space of {ck, Tx} (Fig. 3R). To test whether learned plasticity rules truly required all terms with nonzero coefficients, we compared 'dropout" variants of the discovered rule, in which one coefficient within {cu} was set to zero, its unaltered form. We computed the loss these variants on test networks determine if term's s absence impacted the loss (Fig. 33)). Computing the change in the median loss of networks organized by the learned solution and dropout variant across 14 discovered rules revealed reliable trends in the importance of various terms (Fig. 3p)) We refit these terms progressively in order of the impact on loss across all training runs and found a sharp elbow at 3 terms: a temporally asymmetric Hebbian learning term, ã,æ (blue pentagon in Fig. 33), its complement æąūū (orange square), and a term second order in the postsynaptic activity multiplied by the synapse size, W.yūaar (purple triangle). The first term was consistently learned with positive coefficient while the latter two were
5

---

always almost negative (Fig. 31, rendering the effective learning rule
Wij co c12ūū, و2
(4)
where all {ai} are non-negative and time constants are different for each term. The 3 most important terms be interpreted temporally asymmetric generalization of Oja's rule in that the Hebbian learning term, 2. is replaced by the first two terms in Eq. 4 which depend on the relative timing of Ij and zy. The time constant of the third term was on average short (~1 ms), making this term akin to W.zzy. the normalizing term of Oja's rule.
2.2 Biological noise alters the learned plasticity rules
We next asked whether ongoing disruptions network structure alter which plasticity rules are meta- learned. To explore this, we introduced synaptic turnover to the simulation phase of the meta-learning loop. Synaptic turnover a stochastic process by which existing synapses disappear and new, small synapses emerge (Fig. T)) Prior to each network activation, all connections were updated according to
wigl > 0 and
EST < PST
Wij
wuyl 0 and
ØST 2 PST
Wij
(5)
[iy 0 and ØST < PST
(wsyl = O and EST 2 PST;
where TST U0, 1]. Psr is the probability of single synapse turnover per activation, and is small positive (negative) constant if the presynaptic cell is excitatory (inhibitory). Since plasticity is unable to act on connections of size (see Eq. [I). synaptic turnover determines the set of synapses available to the plasticity rule. To ensure learned rules were robust to a spectrum of rates of synaptic turnover, only half the networks used to evaluate the batch loss underwent this process.
We found that meta-learned rules were able to organize persistent representations of time despite synaptic turnover, with performance near that of rules learned unperturbed networks. Our term- sensitivity analysis (Fig 4A) showed that solutions again heavily depended on temporally asymmetric Hebbian learning, i.e. Jyzty (light blue hexagon), and the bound on postsynaptic activity, wiyāaar (purple triangle); however, we frequently found dependence on two additional terms: one that constantly strengthened all synapses (dark blue pentagon) and an activity bound independent of synapse size (orange square). Refitting the plasticity rule in order of impact on loss demonstrated that these 4 terms recapitulated most of the success of the learned solutions. The effective rule may be written
whi = co + C, - -2 - C
where again all {aj} are non-negative. Synapses for which postsynaptic activity, 2y. remains chron- ically small will be potentiated by co: however, adding this potentiating term also necessitates an activity bound that does not scale with synapse size, such as the term with prefactor in Eq. 6 To understand this impact of this term, consider a synapse between two neurons whose typical firing times are far apart, i.c. H,æy is nearly zero, the fixed point under this rule is
Co
C2
(wyy) max
c3(āErr)) C3
(ājar) > O, where (.) denotes the time average. Thus, a large enough choice of C2 prevents every synapse in the network from growing, enforcing sparsity and decreasing the risk of a neuron changing its firing time upon loss of its original inputs. We additionally note that Eq. dolos not contain - -z,ū,. which appeared in the reduced rule learned in the absence of synaptic turnover (Eq. 4.. This may be because the roles of -z,ū, and -ū,æ, are partially redundant: both terms can suppress synapses that run counter to the sequential dynamics in the network. When constant potentiation of all synapses occurs, the term in žYæ, is preferable as it offsets constant potentiation of all synapses. In the unperturbed context, where constant synaptic growth is unnecessary, the term in ž,æ, is problematic in that it can set all afferent synapses to a driven neuron to zero (whereas - -Wyūjaa cannot). Thus, the term in u. becomes preferable in the unperturbed context.
2.2.1 Comparison to existing models of sequence formation
Do these discovered learning rules more robustly encode timing than previously proposed rules when the circuit is disrupted with biologically relevant noise? We hypothesized this would be true given

---

a
d
10
10
Const
10
X
xy y
330
10*
Vint
10a
Vntt
500
Xint
Loss
Kint
200
innt y
Xint X
300
5
W x
y
100
No. terms included
Loss
200
w y
it
int
100
Wint
0
WXint
Wyint
Summed
Meta leanned
Mete- leamedd with lumover
WXint X
synaptic
Trial 10
Decoder trained
Decode time from neural activity
Figure 4: Discovered rules organized dense feedforward structures (a) Median impact of loss when coefficient of given plasticity is set to zero for a given Iearned solution. (b) Refitting terms in order of median impact on loss shows sequence generation in synaptic turnover context is well captured by 4 terms. (c) Task structure for comparison between different rules. Rules are given 400 activations to organize dynamics. At the end, the decoder is constructed. Networks then undergo synaptic perturbation for 150 activations. Finally, the decoder attempts to decode time from the resulting dynamics to determine the loss. (d) Comparison between multiplicative Hebbian learning and summed synaptic bound, meta-learned without synaptic turnover, and meta-learned with synaptic turnover time encoding task illustrated in (c). Trials shown in grey do not include synaptic turnover; light purple include synaptic turnover. Black crosses indicate median values for each condition.
that discovered rules do not impose hard bounds the size of single synapses, total synaptic strength onto neuron, nor number of synapses, as other models of sequence formation have [1+]3 [7] The absence of these constraints permits compensatory rescaling of synapses in response to disruptions. compared meta-learned rules trained with and without synaptic turnover to a previously proposed sequence learning rule that used multiplicative asymmetric Hebbian learning, single synapse bound, and bounds on the total strength of synapses onto and out of individual neurons [11]. Each rule was applied to 100 test networks for 400 activations without disruption. Following this, decoder was constructed to read out time from the neural activity, and then synapses in the networks were turned during additional 1550 activations, after which the loss evaluated using the constructed decoder (Fig. 4 We compared the best versions of each rule when synapses were turned over (with probability PST 0.00072) each activation, equivalent
90% probability of individual synapse survival during the disruption period (Fig. 41. light purple points), and when they were not turned over (Fig. 4d, grey points; see Supp. Sec. 4). We used the Kruskal-Walln // test equality of medians. We found that meta-learned rules trained with and without synaptic turnover outperformed the rule based based on rigid synapse constraints (p- 2.5 x 10 8 Cohen's d -0.45 and x x 10-7, Cohen's 's d = -0.44. respectively), while in the absence of perturbation, medians were distinct after 4-fold Bonferroni correction (p- 0.016, Cohen's d -0.29 and p 0.029, Cohen's s d -0. -0.30, respectively). When we studied the connectivity structure of networks organized by meta-learned rules, we found that the discovered rule generated denser feedforward connectivity in comparison to other plasticity rules with alternative forms of Hebbian learning and heterosynaptic competition (see Supp. Sec. 5).
for
2.3 Including inhibitory plasticity
Meta-learning allows the exploration of multiple plasticity rules operating on distinct sets of synapses within the same circuit, might arise if there are multiple cell types [35 36) In particular, I-E plasticity has been the focus of much recent work 42. The interaction of many plasticity rules is challenging to analyze theoretically. but meta-learning allows exploration of these interactions Recent evidence suggests there may multiple forms plasticity within sequence- generating circuits: Wang et al. [18) increased the intrinsic excitability of in vivo HVC(RA) neurons and found that the strength both E-II and I-+E connections could dynamically shift response. Targeted cells received increased total inhibitory synaptic strength and decreased excitatory strength.
7

---

?
?
25
rules only
400
All rules
Loss
300
200
100
10
12 16
Turnvee rate
Figure 5: Learning plasticity all synapses (a) Schematic of synapses upon which we newly allow plasticity. (b) Weight matrices on activations and 400 of a network evolving under plasticity rules learned all sets of synapses. synapses increase. (c) Comparison of performance of rules learned only on E->E synapses (red) (N=5) versus all sets of synapses (blue) (N=8) when training includes synaptic turnover across varying rates of synaptic turnover.
a
summed
surneed surrmed
1.5
1.0
1.0
III
0.5
0.5
0.5
0.0
1.5
summed
summed 0.5
Rec.
1.0
Rec
1.0
Rec
0.0
10
20
20 Activations since pert.
Activations since pert.
Activations since pert.
Noree peak ampltuud
Normed peak amplttude
1.75
Normed peak amplitude
1.0
1.50
0.5
1,25
1.00
0.0
0.0
0.5
1.0
Normed duration
Normed duration
Normed duration
Output amplttdd (Hz)
Output ampitude (H2)
Oulput amplltude (Hz)
Oulput ampitudo (Hz))
E
0.8
0.8
0.8
0.8
E
E4.
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.02
0.04
0.02
0.03
0.02
0.04
0.02
0.03
Output duration (s)
Delay (s)
Output duration (6)
Delay (s)
Figure 6: Network perturbations reveal homeostatic compensation in E- and I-E synapses (a) Response of the magnitude of summed E-EE weights | = EI - (top) and summed
recurrent weight W! E w E. w to single cell. Values for neuron with perturbed inputs shown light purple; others black. Each line represents average values for single learned rule over N=20 networks. (b) Same (a), but for imposed upscaling of E-> weights to one cell (light purple). (c) Same as (a), but frequency of stochastic inputs to one cell is greatly increased (light purple) relative all other cells (black), (d-f) Postsynaptic response affected neurons 20 networks plotted in the space normalized peak amplitude and duration for manipulations (a-c). Black dots represent pre-perturbation responses; light purple, immediately after perturbation; blue, 25 activations after perturbation. (g) Schematic of beed-forward motif maintaining homeostasis via its E input.
i,k k, (bottom) to the imposed scaling down of E E weights
(h) Schematic of feed-forward motif (i) Phase flow in space of output firing envelope duration and amplitude (left) for single neuron. Black points first responses of the neuron for various inputs; blue stars represent responses after 200 activations. Note: initial and final output durations and delays roughly the same. Colors of trajectories indicate how the neuron's 's input weight changes to satisfy the plasticity rule: see legend. (j) If the forms of homeostasis on E-+E and I-+E maintain different aspects of the postsynaptic response, attractor forms in the duration and peak amplitude of the postsynaptic response (left). The response delay is also constrained (right).
with homeostasis on E-EE and I-EE synapses.

---

To understand the role of this additional plasticity, we meta-learned plasticity on all sets of synapses within the circuit (Fig. 55p both with and without synaptic turnover. Specifically, we attempted to learn three independent plasticity rules that operated on three distinct groups of synapses (E-+E, E->I, and E->E; see Supplementary Section 6).
2.3.1
E-I and I-EE synaptic plasticity improves decoding of time, particularly in the presence of synaptic turnover
Meta-learning uncovered triplets of plasticity rules that successfully organized initially random networks into sequence generators (Fig. When trained with turnover in E-E synapses, solutions that acted upon E-+I and I-E synapses in addition to E-E outperformed solutions that only acted upon E->E connections, particularly when the rate of synaptic turnover was high (Fig. ) suggesting this additional plasticity played an important role in maintaining network dynamics through perturbation.
To investigate how rules acting on all synapses generated improved time representations, we repeated the dropout analysis. We found that E-EE plasticity within these learned triples was largely similar to the rules previously learned on E->E synapses alone (Eqs. 4 and 6 solutions were consistently sensitive the removal of Ryar, and w W.y which appeared consistently with positive and negative coefficients, respectively. Further, dependence on these terms persisted when we trained networks with turnover on E-+B synapses or I->E synapses (Supp. Fig. 6). As expected, we also found that solutions depended heavily upon terms that acted upon E-xI and I-E synapses, but this plasticity was more difficult to interpret due to increased trial to trial variability in the discovered rules. We found, however, that the E-I plasticity rule consistently depended on the second order presynaptic term 3,I,, which always appeared with a positive coefficient, suggesting that E cells project to inhibitory counterparts with strength that increases with the E cell level of activity. An implication of dependence on this term is that the strength of an E neuron's recurrent inhibition, defined as |WT Σx "i,k [ where i is the index of the E cell and & indexes the I cells to which it projects, might depend on its level of activity. Thus, ablation of excitatory inputs to an E cell might cause its recurrent, disynaptic inhibition to lower in a manner that homeostatically restores its firing.
2.3.2 Recurrent inhibition of E neurons is homeostatic in networks with learned plasticity
As the role of E-II and I->E plasticity was not completely clarified by our perturbations far, next investigated how this plasticity adjusted synapses coupled to an individual E cell when its typical input manipulated. Noting the dependence of the E-II plasticity rule on ,, we hypothesized that a targeted neuron's 's recurrent inhibition and excitatory afferents might be adjusted in concert to restore its typical firing pattern. We used discovered learning rules to organize sequences and then performed three varieties of in silico manipulations of individual E cells within these networks. In the first, we scaled down the excitatory afferents to the targeted E cell by 50% (Fig. [o), diagram). In the second, we scaled up the same connections by 50% (Fig. 66). In the last, we increased the rate of the targeted cell's 's input Poisson process by a factor of 10 (Fig. 66)) This final manipulation mirrored the viral insertion of NaChBac to HVCRA neurons in Wang et al. which causes these cells to become hyper-excitable inhibition and weakened excitatory afferents.
18 45 46
Wang et al. found that manipulated cells recruited additional
We found that scaling down targeted E cell's excitatory afferents resulted in rescaling of those excitatory weights and an accompanying decrease in [Wrec] (Fig. (6)). We further found that these synaptic changes restore the initial firing pattern of the E cell: in Fig Git, we plot the initial responses of targeted E cells across 20 self-organized sequences in the space of duration and peak amplitude of response (black points). Scaling down the excitatory afferents initially causes both the peak amplitude and duration of response to decrease (Fig. 60;; light purple points), but these responses largely recover after 25 activations of the network (Fig. 6 blue points). When we instead strengthened E afferents to the targeted cell, we found these connections weakened and recurrent inhibition strengthened in compensatory fashion (Fig- (6)). In these networks, we found that targeted cell responses that were initially lengthened with increased peak amplitude (Fig. [6: light purple points) were reduced back to their pre-perturbation values of duration and amplitude (Fig. 66; black points show initial responses; blue points, responses after 25 activations). Finally, rendering the targeted E cell hyper-excitable caused it to scale down its excitatory afferents and increase its recurrent inhibition in manner that restored its typical firing pattern. In summary, plasticity rules excitatory afferents and recurrent

---

inhibition operate in tandem to maintain the firing pattern of the neuron (see Supp. Sec. 7 for additional details).
2.3.3
Two forms of homeostasis create an attractor in postsynaptic response and timing
How might modulation of recurrent inhibition contribute differently to the modulation of excitation in preserving network dynamics? We hypothesized that distinct plasticity on different sets of synapses might confer a robust representation of time if these rules governed distinct aspects of the desired network function. For instance, within our HVC-like circuit, peak firing of E neurons might be controlled by E-+E plasticity while total activity might be controlled by I- plasticity. Since excitatory and inhibitory inputs to E cells differ in their timescales (E inputs are transient while imputs are relatively tonic), we reasoned plasticity on both sets of synapses should increase the control of the postsynaptic response. While it is already known that multiple plasticity mechanisms can sharpen the responses of neurons to stimuli (37], prior work has not addressed whether similar plasticity might be leveraged to preserve the timing of such responses, which is crucial to timing representations. To explore this, we constructed a simplified model of a single neuron responding to broad range of excitatory inputs, characterized by varying peak amplitudes and durations, and a tonic inhibitory input. We compared the responses of the neuron when both the excitatory and inhibitory input synapses (Fig. (6)) evolved under plasticity rules responses produced when only the excitatory synapse evolved under a plasticity rule (Fig. (6g] and found the two rule model was able to better constrain the duration, peak amplitude, and timing of the postsynaptic response the E neuron (Fig. 6 -j; see Supp. Sec. 8 for full description of reduced model).
3 Discussion and limitations
Meta-learning plasticity rules via stochastic optimization is a promising technique, but suffers a number of limitations. One, the optimization process becomes expensive as the size of the rule basis, the number of neurons in the network, and amount of simulation time required grows. Further, CMA- ES may require many epochs to converge on good solutions. Training E-EE plasticity (plasticity on all synapses) across 10 networks in batch typically required 24 (72) hours of compute on 30 Cascade Lake or Ice Lake Intel CPU cores to yield reasonable solutions. Two, meta-learning tends to generate different solutions based on the seed; due to the expensive nature of each trial, we did not carry out enough trials to claim full knowledge of the solution space. Three, the plasticity rules learned were quite dense in our choice of basis, limiting interpretation, and we ultimately employed perturbations to better understand the critical terms. Four, the choice of basis limits the space of discoverable rules; for instance, we did not include bodbback-modulated plasticity in this study. Five, the initial connectivity of the circuit likely has a strong bearing on the sort of plasticity that successfully can leverage it 47 48).
Prior to any experience, intrinsic, self- self-organized dynamics within the brain can serve as powerful priors that can accelerate and shape successful, feedback driven learning. In this work, we study how one such computational primitive could emerge by adapting a meta- leta-ne procedure to learn the learning rules that self-organize and maintain robust representations of time in neural dynamics. Meta- learning discovers temporally asymmetric (STDP-like) generalization of Oja's rule that organizes and maintains sparse, sequential activity out of initially random connectivity, which outperforms other models of sequence generation in the presence of synaptic turnover by permitting flexible rescaling of inputs to restore dynamics. Additionally we found that plasticity rules learned on all sets of synapses outperform plasticity rules applied only to E connections. Through a toy model, we show how plasticity on all synapses could confer extra timing stability if the plasticity in distinct sets of synapses act on different moments of an E neuron's 's activity.
In this work, we have developed a paradigm to understand how computational primitives might self-organize within neural circuits and selected sequences as our test example. Future work could study the emergence of other canonical forms of neural dynamics that have been widely identified in brain activity and serve as fundamental components of computation, such as line attractors or limit cycles, or how plasticity rules generating such components interact with rules requiring feedback.
10

---

4 Acknowledgements and disclosure of funding
We would like to thank Carlos Lois, Zsofia Török, Bo Wang, Patrick Zhang, and Leenoy Meshulam for useful discussions. This work was supported by the Simons Collaboration for the Global Brain and an NIH BRAIN grant (SR0INS104925) This work was facilitated through the use of advanced computational, storage, and networking infrastructure provided by the Hyak supercomputer system and funded by the STF at the University of Washington.
References
[1] W Nicola and C Clopath. Supervised learning in spiking neural networks with force training. Nat Commun, 8(1):2208, 2017. ISSN 2041-1723. doi: 1001008/441447-007--118777-
[2] Elias Najarro and Sebastian Risi. Meta-Learning through Hebbian Plasticity in Random Networks, April 2022. /Rtp:t /ttp::///axvvvv.. org/
wwwpaaawwww..../////a.ee/// ht/e//bb//000..... 02686, arXiv:2007.02686
[3] Nikhil X Bhattasali, Anthony M Zador, and Tatiana A Engel. Neural Circuit Architectural Priors for Embodied Control.
[4] Nikhil X. Bhattasali, Venkatesh Pattabiraman, Lerrel Pinto, and Grace W. Lindsay. Neural Circuit Architectural Priors for Quadruped Locomotion, October 2024. URL http:://axxii org/ org/abs/2410. 07174 arXiv:2410001174,
[5] Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matt Botvinick. Learning to reinforcement learn. arXiv preprint arXiv: 1611. 2016.
[6] Pastalkova E, Itskov V, Amarasingham A, and Buzsáki G. Internally generated cell assembly sequences in the rat hippocampus. Science, 3216589)) sep 2008. doi: 10. 10.1//6/cc..nce. 1159775.
[7] Luczak A, Barthó Marguet SL, Buzsáki and Harris KD. Sequential structure of neocortical spontaneous activity in vivo. Proc Natl Acad Sci, 104(1), 2007. doi: 10.1077//pass00005544444444
[8] Gustavo B.M Mello, Sofia Soares, and Joseph Paton. A Scalable Population Code for Time in the Striatum. Current Biology, 25(9):1113-111 may 2015. ISSN 0960-9822. doi: (..01//..cu.2001...0 lisher: Elsevier.
Uttpstsss://///// //doi. ort//o.g///10......111... cub cub. 2015 02. 036 Pub-
[9] R. H. Hahnloser, A. A. Kozhevnikov, and M. S. Fee. An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature, 419(6902):65-700, 2002. ISSN 0028-0836 (Print) 0028-0836. doi: 10. 1018///auur000977..
[10] Ila R. Ficte, Richard H.R. Hahnloser, Michale S. Fee, and H. Sebastian Seung. Temporal sparseness of the premotor drive important for rapid learning in neural network model of birdsong. Journal of neurophysiology, 92:2274 2282, 2004. doi: https://do..org/10. 152/jn. 01133.2003.
[11] IR Fiete, W Senn, CZ Wang, and RH Hahnloser. spike-time-dependee plasticity and heterosy- naptic competition organize networks produce long scale-free sequences of neural activity. Neuron, 65:563-76, 2010. doi:
[12] J Jun and D Jin. Development of Neural Cireuitry for Precise Temporal Sequences through Spontaneous Activity, Axon Remodeling, and Synaptic Plasticity. PLOS ONE, 2(8) August 2007. ISSN 1932-6203. doi: 13. 1/foo7//ooo pon. URL |RL //jpss///jounalls plos. wwtsss.rrppplooe/eeeeeee wtt.ossone/aaticeeeddd///ee//eee 1371/journal. pone. 0000723 Library of Science.
Publisher: Public
[13] P. Zheng and J. Triesch. Robust development of synfire chains from multiple plasticity mech- anisms. Front Comput Neurosci, 8:66, 2014. ISSN 1662-5188 (Print) 1662-5188. doi: 0.3339/fnnom.2044.0006

---

[14] Murray JM and Escola GS. Learning multiple variable-speed sequences in striatum via cortical tutoring. Elife, 6, 2017. doi: 10.7554/cLife.26600
[15] Felix Weissenberger, Florian Meier, Johannes Lengler, Hafsteinn Einarsson, and Angelika Steger. Long synfire chains emerge by spike-timing dependent plasticity modulated by population activity. International Journal of Neural Systems, 27(08): 1750444, 2017. doi:
10.1142/S012906577 29065717500447. PMID: 28982282.
URL https:///oi.orgg// http:p...........11/000000677770
[16] U. Pereira and N. Brunel. Unsupervised learning of persistent and sequential activity. Front Comput Neurosci, 13:97, 2019. ISSN 1662-5188 (Print) 1662-5188, doi: 10.3389/fncom.2113 00097.
[17] Yevhen Tupikov and Dezhe Z Jin. Addition of new neurons and the emergence of a local neural circuit for precise timing. page 43, 2021.
[18] B. Wang, Z. Torok, A. Duffy, D. G. Bell, S. Wongso, T. A. F. Velho, A. L. Fairhall, and C. Lois. Unsupervised restoration of complex learned behavior after large-scale neuronal perturbation. Nat Neurosci, 2024. ISSN 1097-6256. doi:
[19] Y. Bengio, Bengio, and J. Cloutier. Learning a synaptic learning rule. ii:969 vol.2-, 1991. doi: 10.1 10.1109/JCNN, 1991. 155621.
[20] N. Shervani-Tabat and R. Rosenbaum. Meta-learning biologically plausible plasticity rules with random feedback pathways. Nat Commun, 14(1): 1805, 2023. ISSN 2041-1723. doi:
10. 1078//441
[21] D. Tyulmankov. G. Yang, and L. Abbott. Meta-learning synaptic plasticity and memory addressing for continual familiarity detection. Neuron, 1 (0(3):544-557.e0 2022. ISSN 0896- 6273 (Print) 0896-6273. doi: me0oo)..euroo.222..1.1
[22] Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Geesei. On the optimization of synaptic learning rule. In Optimality in Biological and Artificial Networks?, pages 265-287. Routledge, 2013.
[23] Jack Lindsey and Ashok Litwin-Kumar, Learning to Learn with Feedback and Local Plasticity. In H. Larochelle, M. Ranzato, Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 21213-21223, Curran Associates, Inc., 2020. URL f29 222266ee22B564/33556662222226620
www.eddiiinnnnnnnnnnseeeee neurips.....e cc/ wwtppppr_aaeeeeeeeeeeeeee filesr http/pppr..lles//pe////00//////0 /ppeer/020//f1le/
[24] Luke Metz, Niru Maheswaranathan, Brian Cheung, and Jascha Sohl-Dickstein. Meta-Learning Update Rules for Unsupervised Representation Learning, February 2019. http://a/ar
org/abs 1ataaa//18/4... 00222 arXiv:1804.0022.
[25] Thomas Miconi, Jeff Clune, and Kenneth O. Stanley. Differentiable plateicc::blle training plastic neural networks with backpropagation, July 2018. URL http:://axxivoorgaatta haarg//bb/18844 02464 arXiv:1804.02464.
[26] Hector Garcia Rodriguez, Qinghai Guo, and Timolcon Moraitis. Short-Term Plasticity Neu- rons Learning to Learn and Forget, June 2022. arXiv:2206.14044
wttp://www..rr//////2////// 14048
[27] Michael A. Long, Dezhe Z. Jin, and Michale S. Fee. Support for a synaptic chain model of neuronal sequence generation. Nature, 468(7322) November 2010. ISSN 0028- 0836. 1476-4687. doi: 10. (0.1038/nature095114 URL hnttp://wwee www.ra. com/ wtt.urt.co//articeeeee.......eee articces/ nature09514
[28] F. W. Moll, D. Kranz, A. Corredera Asensio, et al. Thalamus drives vocal onsets in the zebra finch courtship song. Nature, 616: 132-136, 2023, doi: 10.003884415886-023-050811 URL
https: //dos: org/ org/10.
12

---

[29] H. H. Danish, D. Aronov, and M. S. Fee. Rhythmic syllable-related activity in songbird motor thalamic nucleus necessary for learned vocalizations. PLoS ONE, 12(6))e0165668, 2017. doi: 10.1371 /ooeural...ooo..0 016566..
[30] G. Kosche, D. Vallentin, and M. A. Long. Interplay of inhibition and excitation shapes premotor neural sequence. J Neurosci, 35(3):1217227, 2015. ISSN 0270-6474 (Print) 0270-
6474. doi: 10. 00.1122/jneurossci.44444.. 434.44.....411111111 14.2015.
[31] Barish Poole, Jeffrey E. Markowitz, and Timothy J. Gardner. The Song Must Go On: Resilience of the Songbird Vocal Motor Pathway. PLoS ONE, 7(6):e38 173, June 2012. doi: 10.1371/
journal.pone.0035. 0038 173. URL patttss://pm... ncbi nlm. nih. www........//atttciee.....eeoeee htt.g///arrriicee/////3///11111
[32] Richard Mooney and Jonathan F. Prather. The HVC Microcircuit: The Synaptic Basis for Interactions between Song Motor and Vocal Plasticity Pathways. Journal of Neuroscience, 25(8):1952-1964, 2005, ISSN 0270-6474, doi: 10.
WWW.RROOSI...26-4..... URL Publisher: Society for Neuroscience _eprint: http:://wwww..puuuss....oooooooa
htttp:://www.jjuurrrr org /ttp:/aac..aaaeeoootee//////////
1952
[33] Basile Confavreux, Friedemann Zenke, Everton J. Agnes, Timothy Lillicrap, and Tim P. Vogels. meta-learning approach to (re discover plasticity rules that carve a desired function into neural network. In 34th Conference on Neural Information Processing Systems, Vancouver, Canada, 2020.
[34] Anne Auger and Nikolaus Hansen. A restart evolution strategy with increasing population size. volume 2, pages 1769-1776, 01 2005, doi: (LL09/CCEC2220555549999
[35] Manuel A. Castro-Alamancos and Barry W. Connors. Distinct forms of short-term plasticity excitatory synapses of hippocampus and neocortex. Proceedings of the National Academy of Sciences, 94(8):41614416,, April 1997. doi: (0073//nas.94.8.444 URL https:// Publisher: Proceedings of the National
www www.pnas. org/ wwrrg//oo/11..o 10wp://11.1001/11111 1073/ wwt.:/003.........m. pnas. 94.894.8.. 4161
Academy of Sciences.
[36] Rylan S. Larsen and P. Jesper Sjöström. Synapse-tye-speecffe plasticity in local circuits. Current opinion in neurobiology, 35:127. August 2015. doi: (o0l://conbb2015.000 URL https: /ttps::/pmc... ncbi. nlm. nih. gow/
[37] T. P. Vogels, H. Sprekeler, F. Zenke, C. Clopath, and W. Gerstner. Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science, 334(6062):
1569-73, 2011. ISSN 0036-8075. doi: 10.1 126///ience. 1211099.
[38] Ashok Litwin-Kunnn and Brent Doiron. Formation and maintenance of neuronal assemblies through synaptic plasticity. Nature communications, 5(1): 2014.
[39] Yotam Luz and Maoz Shamir. Balancing Feed-Forward Excitation and Inhibition via Hebbian Inhibitory Synaptic Plasticity. PLOS Computationall Biology, 8(1):1-12, Jamuary 2012. doi: 10.1371/journal pob.000033....0003......0 1002334. URL http://t::///ii.rr//101/11//////
wttp:lccci.......... 1002334
Publisher: Public Library of Science.
[40] Waitzmann F, Wu YK, and Gjorgjieva J. Top-down modulation mod-lattmoootttion canonical cortical circuits with short-term plasticity. Proc Natl Acad Sci, 121(16), 2024. doi: 10. 10.1073/pnas.331 1040011..
[41] Ziyi Gong and Nicolas Brunel. Inhibitory Plasticity Enhances Sequence Storage Capacity and Retrieval Robustness, April 2024. URL https::p// http:://wwwwwwwwww..oa/ biw.xiv..ivv.... wwwoor/////g///oo/oeeeoo
2024. 04. 08. 588573v1 Pages: 2024.04.08.58882
Section: New Results.
[42] Fereshteh Lagzi and Adrienne L.. Fairhall. Emergence of co-tuning in inhibitory neurons as network phenomenon mediated by randomness, correlations, and homeostatic plasticity. Science Advances, 10(12)ead March 2024. ISSN 2375-2548, doi: 10. 10.1126/sciadv..dd44350 URL
https::/www. sttps://sww.scieencee
science. wwwg/////////o/o/o///oo//oo/ooog http////oo...11//ccacaa..a. wtt.1112//sciaae wwtp:/iiiinnee...cooooooeeoooeoe adi4350
[43] Basile Confavreux, Everton J. Agnes, Friedemann Zenke, Henning Sprekeler, and Tim P. Vogels. Balancing complexity, performance and plausibility to meta learn plasticity rules in recurrent spiking networks, June 2024. | : w/www./www//c wtwppwwww.i...owwwttt/twwwttpttp .or.vv.orrggaooo wtt.//onnntttno1100/00
06. 17. 599260v1
Pages: 2024.06.17.599222
Section: New Results.
13

---

[44] Basile Confavreux, Poomnima Ramesh, Pedro J Gonçalves, Jakob H Macke, and Tim P Vogels. Meta-learning families of plasticity rules recurrent spiking networks using simulation-based inference.
[45] Shuyin Sim, Salome Antolin, Chia-Wei Lin, Yingxi Lin, and Carlos Lois. Increased cell-intrinsic excitability induces synaptic changes in new neurons in the adult dentate gyrus that require npas4. Journal of Neuroscience, 33(18):7928-7944 2013. ISSN 0270-6474 doi: 10.1523. IEEURSSCL1571112.2000 URL htLLhttps:/wwet jnw.rosciioccccoo http:/ooneett///1/////
[46] C. W. Lin, S. Sim, A. Ainsworth, M. Okada, W. Kelsch, and C. Lois. Genetically increased cell-intrinsic excitability enhances neuronal integration into adult brain circuits. Neuron, 65(1):
32-39, jan 2010. doi: 10.
12.001.
[47] Ashok Litwin-Kumar, Kameron Decker Harris, Richard Axel, Haim Sompolinsky. and L. F. Abbott. Optimal Degrees of Synaptic Connectivity, Neuron, 93(5):1153 1164.e7, March 2017.
ISSN 1097-4199. doi: 10.
[48] Kaushik J. Lakshminarasimhan. Marjorie Xie, Jeremy D. Cohen, Britton A. Sauerbrei, Adam W. Hantman, Ashok Litwin-Kumar, and Sean Escola. Specific connectivity optimizes learning in thalamocortical loops. Cell Reports, 13(4):114055 April 2024. ISSN 2211-1247. doi:
10. 10.10166..cerep22224.1444 114059.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The primary claims of the paper are addressed computationally in Section results, and are expanded upon in the supplement.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
Justification: The limitations of the findings and methods are discussed extensively in the Discussion and Limitations section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [NA]
Justification: The paper sketches several theoretical ideas, but does not make rigorous theoretical claims, instead opting to illustrate phenomena via computational approaches.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Code referenced in supplementary section, Code Availability.
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
Code referenced in supplementary section, Code Availability.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
7/nips.com wttp:ccppppcccccee. guwde//dde/eeee Cooeeeoodeesssseeeeeee http://dd///uuudddd/doo/////e/e/
ssioislicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: The paper clearly outlines all methods. See section: Results, and also supple- ment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Section 2, results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computational resources used are detailed in the Discussion section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The NeurlPS code of ethics is fully abided by in this work.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Societal impacts are expounded upon in the discussion section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Code, data, and models used in the study belong to the authors or are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All new assets well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No human subjected were used in this study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines: No IRB approvals were required for this study.
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "iD7qBNOzMp", "reviewer": "Reviewer_JAan", "summary": "The paper proposes a conceptually simple set-up to learn plasticity rules that result in neural networks that perform a specific task. The set-up of the paper is elegant and made up of simple but effective elements, such as linear-threshold neurons and evolutionary optimisation. The authors obtain some intriguing results, e.g. that their method discovers plasticity rules well-known in experimental neuroscience.", "questions": "The p-values in Sec. 2.2.1 are somewhat meaningless, because one can always run more simulations to decrease the p-value. I would recommend quantifying effect size (with e.g. Cohen's $d$) instead.\n\nPlease define important terms at first use (e.g. HVC).\n\nWhat is $\\mathcal{P}(\\textbf{C})$ in L. 103?\n\nReferences 4, 5, 11 and 28 are misformatted.", "limitations": "The \"Discussion and limitations\" section is very short and should be expanded.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper tackles an interesting problem with a relatively straightforward set-up, which makes it easily understandable. It has a very high density of results, which are technically sophisticated as well as relevant for many questions in neuroscience.", "weaknesses": "One of the main limitations of the paper is that this very promising and flexible set-up is only applied to one very idiosyncratic task of elapsed time decoding. This is a very specific task, and not one that brains do spontaneously (see next paragraph), so although it yields interesting results it seems a bit lacking. It would be tremendously useful to see the results in a broad range of tasks, to see e.g. under what conditions other common plasticity rules (Hebbian, anti-Hebbian, perhaps some dopamine-like reward-modulated plasticity) emerge.\n\nThe motivation of the system set-up is somewhat dubious. The authors point out the importance of \"intrinsic, self-organised dynamics\" but then include as the primary component of their loss function the performance of an external decoder that predicts simulation time. Neither the decoder nor the ground truth time values are in any way intrinsic to neural activity, so I would argue this is not representative of self-organisation.\n\nOn a separate note, the paper sorely lacks any demonstration of the \"downstream\" effects of these plasticity rules. For example, the author's motivation is that \"self-organized computations provide scaffolds for solving difficult tasks\". However, the authors don't actually show that the networks evolved with their discovered plasticity rules can actually solve any difficult task.\n\nThe paper gets really hard to follow towards the end, particularly Sections 2.3.2 and 2.3.3. The intervention to increase or decrease E afferents in Sec. 2.3.2, as well as the analysis metrics in Fig. 6a-c aren't very clear. It also surprising to see Sec. 2.3.3, the final results subsection of the paper, introducing a whole new model and a different plasticity rule, which are only partially related to the model and plasticity rules studied elsewhere. Overall, I could not understand well the set-up or results in either of these subsections.\n\nThe details of the meta-learning procedure should be explained more thoroughly in Sec. 2.1.1. Figures are quite busy and in general hard to parse.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful reading of our paper and their comments.\n\nWe agree with the reviewer that this promising methodology should be used to study the self-organization of other computational motifs, such as line and ring attractors, and we hope that this paper provides a path for doing so. We have added a line in the introduction highlighting this potential future direction. It reads:\n\n“Though the results we present in this work are specific to sequence generation, our methodology might be applied to other basic circuit motifs such as line and ring attractors.”\n\nWe also agree the approach should be expanded in the future to include different types of plasticity, including feedback-based rules, while this is not entirely new: Shervani-Tabar and Rosenbaum (Nat. Commun., 2023) learn feedback-based, biologically plausible rules for deep network training. We view our work as a proof of principle for studying the self-organization of robust computational motifs.\n\nWe agree that neurons in general have no knowledge of ground truth simulation time. Time in our simulation is locked to the time of an input. We have clarified in our methods that we divide the simulation of each plasticity rule into 400 activations of 110 ms. At t = 10 ms of each activation, the initial kick of excitation is presented to the network. Since this stimulus is time-locked to the simulation clock, attempting to decode time elapsed since the beginning of the activation is the same as decoding the time elapsed since stimulus presentation. Thus, our networks track the time since the inciting pulse initiates the dynamics, and the decoder is merely a means by which to evaluate the success of the plasticity rule at accomplishing this.\n\nWe have rewritten section 2.1.1 to better describe the meta-learning procedure, as requested. It now reads:\n\n“We next define a loss function that evaluates the quality of the sequential dynamics organized under a chosen $\\\\{c\\_k, \\tau\\_k\\\\}$, and attempt to minimize it using an evolutionary strategy, Covariance Matrix Adaptation (CMA-ES)\\cite{auger_and_hansen}. CMA-ES places a multivariate Gaussian on the space of the parameters to be optimized, then samples from that Gaussian and evaluates the fitness at each point. The fitness is then used to adjust the mean and covariance matrix of the distribution in an attempt to iteratively maximize fitness. We use CMA-ES to sample from the space of possible $\\\\{c\\_k, \\tau\\_k\\\\}$. We evaluate the loss for each choice of values $\\\\{c\\_k, \\\\tau\\_k\\\\}$ defining the plasticity rule by simulating 10 randomly initialized networks under this rule and evaluating the resultant dynamics at the end of the simulation. Each simulation consists of 400 activations of 110 ms. At $t = 10$ ms of each activation, an arbitrary neuron is driven by a strong kick of excitation \\cite{tupikov_2021_addition}. Following this, all other neurons in the network receive Poisson distributed input for a period of $100$ ms. We generate the Poisson inputs as a sum of a component which is generated randomly trial by trial, and a frozen component, held fixed from trial to trial, which we take to model song-locked input to HVC from the nucleus Uva \\cite{Murray_2017}. The total loss for a given rule is simply the sum of losses across each of the 10 networks.”\n\nWe have attempted to declutter figures and enlarge the size of figure labels and text, where possible.\n\nWe have also clarified sections 2.3.2 and 2.3.3 to more clearly describe our procedure and results in perturbing networks organized under joint excitatory and inhibitory plasticity.\n\nAs requested, we have added a quantification of Cohen’s $d$ to Sec. 2.2.1. We found that the learned rules have a moderate to large effect on the loss when compared against the rule from the literature. We report this in the text as follows:\n\n“We used the Kruskal-Wallis $H$ test to test for equality of medians. We found meta-learned rules trained with and without synaptic turnover outperformed the rule based based on rigid synapse constraints ($p=2.5 \\, \\mathrm{x} \\, 10^{-8}$, Cohen's $d = -0.45$ and $p=7.2 \\, \\mathrm{x} \\, 10^{-7}$, Cohen's $d = -0.44$, respectively), while in the absence of perturbation, medians were not distinct after 4-fold Bonferroni correction ($p=0.016$, Cohen's $d = -0.29$  and $p = 0.029$, Cohen's $d = -0.30$, respectively).”\n\nWe have added a definition of HVC, which historically was called the High Vocal Center, but is now simply used as a proper noun. We include a description of HVC physiology in the response to R2.\n\nWe have also amended $\\mathcal{P}(\\textbf{c})$ on L. 103 to $\\mathcal{P}(\\textbf{c}, \\tau)$ (note: $\\tau$ should be bolded as well). $\\mathcal{P}(\\textbf{c})$ has no definition; this was a typo.\n\nWe thank the reviewer for pointing out our misformatted references.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xM4Ku3U61Y", "reviewer": "Reviewer_rSVd", "summary": "This paper proposes a method for discovering plasticity rules in spiking neural networks to achieve sequence generation using both excitatory and inhibitory dynamics. Rules are parametrized with basis function. The biological approach in the model involves also considerations homeostasis and robustness to perturbation. The HVC region of the zebra finch was used as inspiration. The study shows interesting finding, including a self-discovered generalisation of the Oja rule, robustness to perturbation and enhanced stability, including homeostatic mechanisms.", "questions": "- How many fitness evaluation through CMA-ES were performed? How much time did one evaluation take? How did the fitness evolve over time? \n- Following up from a point in the weakness. How can these findings be used? Is there a recommendation that we should use evolution to implement and tune artificial spiking neural networks?", "limitations": "The paper addresses the limitations, in particular in relation to computational demands.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The method uses interesting biological concepts, including both excitatory and inhibitory signals, and local Hebbian-like rules.\n- The study analyses various properties of the evolved neural circuits, including robustness to perturbation and robustness to synaptic turnover.\n- The model includes the use of noise, an important factor that affects both biological and artificial systems. \n- The computational study supports the idea, not new in biology, that both excitatory and inhibitory synapses are important in maintaining equilibrium.\n- The method allows for testing of particular hypothesis, e.g., the utility of dense feedforward connectivity.", "weaknesses": "- Unclear novelty: there have been many studies that evolved plasticity rules using some form of evolutionary computation and biological plausible rules. In fact, evolution of Hebbian-like rules has been a popular area of research for decades. While this study advances many previous studies, I feel that the novelty and contributions in this area are not well highlighted with respect to existing studies. \n- It is unclear to me how the findings can be used more broadly. I agree that it is interesting the CMA-ES and meta-learning can optimize plasticity rules to have interesting biological properties, but how can this finding be used? This is the most important concern I have: what impact can this paper and its discoveries have in the field? \n- Figures' clarity: there are a lot of assumption on the meaning of symbols, colours, etc, in the figure. E.g., Fig 1c, unless one is very familiar with the zebra fish, I don't think these graphs very useful. What is the big blue dot on top of the red ones? what do the colour in the cell-index vs. time mean? what does the graph mean? \n- The manuscript describes synaptic turnover starting from the abstract, but a search of the term to find out what that is and how it is implemented did not provide satisfactory results. \n- There may be an issue with the submission as the Supplementary information is not included, resulted also in broken references (e.g. line 197, page 7)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful consideration of our manuscript.\n\nWe address the reviewer’s concerns regarding novelty in the common response. Briefly, our unique contribution is the exploration of unsupervised and unrewarded plasticity via meta-learning that organizes and maintains a specific and biologically relevant computational motif, a sequence. We have clarified this in the introduction. While prior work on sequence self-generation used hand-built rules, we learn the optimal plasticity rule by maximizing its ability to represent time within the network dynamics. Further, we believe the inclusion of biological noise, such as instability in synaptic connectivity, in a learning procedure is novel, particularly in the sequence generation context.\n\nRegarding broader impact, our work suggests a specific role for unsupervised plasticity: the establishment and maintenance of simple computational motifs. We hypothesize that the plasticity that organizes these computational motifs might be extremely specific to the structure being organized and may altogether lack feedback modulation. This suggests we should look for plasticity rules that might underlie the self-organization of other common computational motifs, such as line and ring attractors. From the standpoint of experimental predictions, this work provides a hypothesis for why behaviors like path integration may be innate to or rapidly learned by some species. It additionally suggests that such simple computation motifs in the brain may lack dopaminergic or other reward-related inputs.\n\nWe apologize for the unclear definition of synaptic turnover: by this we mean the disappearance of existing synaptic connections and emergence of nascent ones over time. We now explicitly define it, and include an equation in Sec. 2.2. Our addition reads:\n\n“We next asked whether ongoing disruptions to network structure alter which plasticity rules are meta-learned. To explore this, we introduced synaptic turnover, a stochastic process by which existing neurons disappeared and new, small synapses emerged, to the simulation phase of the meta-learning loop. Prior to each network activation, all connections were updated according to\n$$    w_{ij} \\\\leftarrow \\\\begin{cases} \n      0 \\\\quad \\rm{ if } \\\\quad  |w_{ij}| > 0 \\\\; \\\\mathrm{and} \\\\; x_{\\rm ST} < p_{\\rm ST}, \\\\\\\n      \\\\; \\\\;w_{ij} \\\\quad \\rm{ if } \\\\quad  |w_{ij}| > 0 \\\\; \\\\mathrm{and} \\\\; x_{\\rm ST} \\geq p_{\\rm ST}, \\\\\\\n      \\\\; \\\\;\\epsilon \\\\quad \\rm{ if } \\\\quad |w_{ij}| = 0 \\\\; \\\\mathrm{and} \\\\; x_{\\rm ST} < p_{\\rm ST},\\\\\\\n      \\\\; \\\\; 0 \\\\quad \\rm{ if } \\\\quad  |w_{ij}| = 0 \\\\; \\\\mathrm{and} \\\\; x_{\\rm ST} \\geq p_{\\rm ST}\\\\}\n    \\\\end{cases}\n$$\nwhere $x_{ST} \\sim U[0, 1]$, $p_{ST}$ dictates the rate of synaptic turnover, and $\\epsilon$ is a small positive (negative) constant if the presynaptic cell is excitatory (inhibitory). Since per Eq. 1, synaptic turnover determines the set of synapses available to the plasticity rule since the rule is unable to act on connections of size 0. To ensure learned rules were robust to a spectrum of rates of synaptic turnover, only half the networks used to evaluate the batch loss underwent this process (Fig.1f, bottom).\" \n\nNote, OpenReview does not seem to render \\begin{cases} within equations correctly.\n\nWe have added the following description of the computational resources used for this study to the discussion:\n\n“Training E$\\rightarrow$E plasticity across 10 networks typically required 24 hours of compute on 30 Cascade Lake or Ice Lake Intel CPU cores to yield reasonable solutions. Training plasticity across all sets of synapses as in Sec. 2.3 required up to 72 hours across the same number of cores.”\n\nFollowing the reviewer’s suggestion, we have added a section with background on zebra finch physiology. This section reads:\n\n“In the zebra finch, premotor nucleus HVC contains excitatory neurons that fire sparsely (typically in one burst of spikes) during song and are purportedly arranged in a feed-forward structure (Fig. 1c). A subset of these cells, known as $\\\\mathrm{HVC}\\_{\\rm (RA)}$ neurons, project to downstream nucleus RA (robust nucleus of the archistriatum), which in turn projects to vocal neurons of the syrinx and to the brainstem, which regulates respiration \\cite{Hahnloser_2002}. HVC additionally receives excitatory projections from nucleus Uva, which controls the onset of song syllables \\cite{Moll2023} and provides input for the duration of song \\cite{Danish2017}. $\\mathrm{HVC}_{\\rm (RA)}$ neurons inhibit each other disynaptically via a population of inhibitory interneurons within HVC \\cite{Kosche_2015}.”", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cXLCjUD8or", "reviewer": "Reviewer_4Ykq", "summary": "This work applies a meta-learning procedure for plasticity rule discovery to a neural network model of sequence generation. Plasticity rules are constructed from parameterized basis functions, and the parameters are found through evolutionary search based on a fitness function quantifying how accurately the plasticity rules train the network to represent elapsed time within episodic rollouts. The work analyzes the plasticity rules discovered in a variety of conditions: with noise, with synaptic turnover, and with plasticity enabled on excitatory and/or inhibitory neurons.", "questions": "See above.", "limitations": "Yes, the authors adequately address the limitations in the Discussion section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Significance**: Hand-tuning plasticity rules can be tedious and suboptimal, so automated methods are important for the more efficient and effective design of models. By applying this method to a neural network model of sequence generation (specifically, a model of HVC), this work tests the procedure in a new setting and also develops a more robust plasticity rule for sequence-generating dynamics.\n\n**Novelty**: It seems the novelty is in the application of this method to a HVC model and the evaluation of the learned plasticity rules under a variety of conditions. If there are other contributions, it would be helpful to be clearer about contextualizing this in the related work.\n\n**Technical quality**: This work provided an extensive evaluation of the model in a variety of conditions. The analysis of the discovered plasticity rules was particularly interesting.\n\n**Presentation clarity**: The methods and results are clearly explained.", "weaknesses": "**Wiring vs plasticity**: It seems like an assumption of this work is that the sequence generating structure of HVC neural circuits emerges largely due to activity-dependent plasticity in a network with initially random connectivity. What is the role of activity-independent wiring that takes place during development? And how much of an effect does more structured initialization have on the discovered plasticity rules?\n\n**HVC predictions**: As a main contribution of this work is improving a model of HVC, what experimental predictions will result from this computational analysis?\n\n**Figures improvements**: The figures in general have pretty small text, inconsistency in formatting, and subpanels that are cramped. Fig 1 especially has a lot going on, uses a variety of fonts, and adopts a colorblind-unfriendly palette. Fig 2c, I didn't see why the 3 trials were needed; could it just show one and put other trials into the Supplementary? Fig 4d, use 45deg or 90deg xlabel angles instead?\n\n**Text improvements** (minor): (1) The neuron model notation on Line 77 took me a while to parse and seems non-standard according to colleagues; ultimately, it is a simple exponential decay and such neural dynamics are usually expressed in dv/dt notation. (2) The spacing on Page 11 is messed up due to the type of latex linebreak used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful comments. We have worked to address their suggestions and make changes to the manuscript accordingly.\n\nWe agree with the reviewer that activity-independent wiring plays an important role in determining the capacity of a network to self-organize (see Lakshminarasimhan et al, 2024, Litwin-Kumar, 2017). Tyulmankov et al, 2022, also learned Hebbian-like rules via a supervised learning strategy, but first optimized connectivity in their networks via backpropagation. Our study makes a few assumptions about initial connectivity, e.g. we do not interconnect interneurons as this connectivity is very rare in HVC, but we did not explore, for instance, how the sparsity of different types of connectivity might affect the learning rules. We now note this in the discussion section, saying:\n\n“... the activity-independent initial connectivity of the circuit likely has a strong bearing on the sort of plasticity that successfully can leverage it \\cite{litwin_kumar, lakshminarasimhan_2024}. For instance, sparsity is known to affect the efficacy of supervised versus unsupervised plasticity. We did not explore this important facet while learning plasticity rules, though in principle it could be included as part of the optimization.”\n\nWe have attempted to clarify our contributions beyond the HVC model and evaluation of learned plasticity under synaptic turnover, as noted in the overall response. \n\nWe have also clarified the experimental predictions that stem from our improved model of HVC. The work makes an explicit prediction for the nature of the learning rule in HVC that could be tested using methods to infer learning rules from data (Pereira and Brunel, 2018). Further, the hypothesis that plasticity on inhibitory and excitatory synapses might maintain different aspects of HVC(RA) cell firing leads to several predictions: (1) inhibition plays an essential role in maintaining the firing profile of HVC(RA) neurons, (2) manipulating the firing rate envelope of an HVC(RA) cell should cause the neuron to change both its excitatory and inhibitory inputs.\n\nWe have simplified Fig. 1 in accordance with the reviewer’s comments: we removed the right portion of Fig. 1C and provided a more useful caption for the network diagram. We have additionally modified Fig. 2c to only include one trial. We have also standardized fonts and font sizes in all figures. We have further improved figure clarity by ensuring all symbols and colors are properly defined.\n\nWe have rewritten the equation on L77 as suggested. It now reads:\n\n“Each neuron fired according to $x\\_{j}(t) = \\\\left[ V\\_j(t) - b \\\\right]^+,$ where $V\\_j(t)$ evolves via $\\\\tau\\_m \\\\dot{V}\\_j(t) = -V\\_j(t) + \\\\sum\\_i w\\_{ij} x\\_i(t)$.”\n\nWe thank the reviewer for catching the spacing issue on page 11.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vdu1Jx2mq5", "reviewer": "Reviewer_6oGR", "summary": "The authors investigate the development and persistence of intrinsic connectivity structures within a\nneural region without external input. They propose that a local plasticity rule can lead to self-organising connectivity motifs, creating inductive biases beneficial for subsequent information processing. The study focuses on sparse and sequential dynamics, as observed in the HVC region of zebra finches. The paper explores which plasticity rules can organise and maintain these sequential dynamics using meta-learning.The space of plasticity rules is parameterised using terms based on activations and synapse size.\nThe coefficients and their time constants are adjusted to minimise the loss, which comprises three\ncomponents: encouraging sparsity in activation, synaptic change, and improving the accuracy of time decoding. Initially, excitatory-excitatory rules are considered, followed by inhibitory-excitatory and excitatory-inhibitory rules. The study evaluates the efficiency of these rules under synaptic turnover and compares them to established Hebbian plasticity rules. Evolutionary algorithms identified a rule resembling a temporally generalised Oja’s rule, which\noutperformed a multiplicative asymmetric Hebbian rule in the context of synaptic turnover.", "questions": "- Why does the second term, which was estimated without noise, lose its importance when noise\nis introduced? The authors did not address this point.\n- Why did the authors not explicitly explore the interaction between the estimated plasticity rule and\na subsequent supervised learning procedure, despite strongly hinting at this in the introduction?\n- Are networks using the estimated rule more effective at reproducing songs than those employing a\ntraditional Hebbian rule?\n- Why was the decoding of time not reported throughout the paper, given that the reduction of loss\ncould be influenced by three different factors?\n- Could you clarify why the estimated rule is considered a temporally generalised Oja’s rule?", "limitations": "The authors were aware of the limitations of the methods used in the paper and addressed\nthem accordingly.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is conceptually significant, presenting the novel idea that local synaptic plasticity\ncould be crucial for maintaining inductive biases in the form of intrinsically-maintained\ncomputational motifs. These internally generated dynamics must interact with synaptic\nchanges induced by supervised learning—they need to support this form of learning without\nbeing overwhelmed by it. This perspective introduces a fresh angle to the\nunderstanding of the function and origin of local plasticity rules.\nThe authors primarily focus on identifying synaptic rules that can create stable inductive\nmotifs resilient to perturbations. The analyses presented are thorough and convincingly\nsupport this point. The use of meta-learning to estimate these plasticity rules is an innovative\napproach. Additionally, the detailed examination of the estimated rule offers an interpretable\noutcome from a complex machine learning technique - a rare occurrence in the field.\nA particular strength of the paper is its exploration of not only E-E but also E-I and I-E rules.\nFurthermore, the authors again analyse the estimated rule in a manner that provides interpretable\noutcomes.", "weaknesses": "The manuscript occasionally feels unfinished and rushed, particularly in section 2.3.1.\nThe references to Figure 3 (blue circles, green stars?) and the lack of references to Figure 4\nin the text were often frustrating. Additionally, I was keen to understand why the second term\nestimated without noise lost its significance when noise was introduced, but the authors did\nnot address this point.\nFurthermore, it was somewhat anticlimactic that the authors did not explicitly explore the\ninteraction of the estimated plasticity rule with subsequent supervised learning, as this was\nstrongly hinted at in the introduction. For instance, are networks with the estimated rule better\nat reproducing songs than models with a traditional Hebbian rule?\nLastly, I found it odd that the decoding of time was not reported throughout the paper. Since\nthe reduction of loss can be attributed to three factors, this omission is noteworthy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful evaluation of our manuscript. We have worked to address all comments and rewrite our paper accordingly.\n\nWe now provide a rationale for why the second term in Eq. 4 loses its significance in the perturbed context. We have added the following to Sec. 2.2:\n\n“We additionally note that Eq. 6 does not contain $-x_i \\tilde{x}_j$, which appeared in the reduced rule learned in the absence of synaptic turnover (Eq. 4). We expect this is because the roles of  $-x_i \\tilde{x}\\_j$ and $- \\\\tilde{x}\\_j x\\_j$ are partially redundant: both terms can suppress synapses that run counter to the sequential dynamics in the network. When synapses are turned over, the latter term is preferable as it offsets constant potentiation of all synapses. In the unperturbed context, constant synaptic growth is unnecessary, and therefore $- \\\\tilde{x}\\_j x\\_j$ is problematic in that it can set all synapses to a driven neuron to zero (note $-w\\_{ij} \\\\tilde{x}\\_j x\\_j$ cannot do this). Thus, $-x\\_i \\\\tilde{x}\\_j$ becomes preferable.”\n\nAs we note in the common response to all reviewers, we chose to focus on the self-organization of the timing scaffold as the utility of a timing representation to support motor learning has already been well studied (e.g. Duffy et al. PNAS 2019). See common response for full details.\n\nRegarding comparison between the estimated rule and a traditional Hebbian rule, Hebbian plasticity that only potentiates coincident activity (as found in a Hopfield network) cannot be responsible for the organization of sequential activity since asymmetry in the connectivity is required; traditional Hebbian plasticity tends to foster symmetric connectivity (see Sompolinsky and Kanter, 1986). We therefore chose to compare the estimated rule to an existing rule previously used in the literature in the context of this system (Fiete et al., 2010) which employs a temporally asymmetric form of Hebbian learning that is similar to that of our estimated rule, but imposes a sum on the total synaptic strength onto and out of individual neurons. We compare the estimated rule to this pre-existing rule in Fig. 4d.\n\nWe have now reported the accuracy of time decoding in Fig. 2. For all rules, the dominant contribution to the loss is the accuracy of time decoding, reported as $1000(1 - R^2).$\n\nWe have clarified the sense in which the rule estimated in the unperturbed context on E$\\rightarrow$E synapses is a generalization of Oja’s rule. We have added the following to Sec. 2.1.3:\n\n“The 3 most important terms can be interpreted as a temporally asymmetric generalization of Oja's rule in that the Hebbian learning term, $x\\_i x\\_j$, is replaced by the first two terms in Eq. 4, which depend on the relative timing of $x\\_i$ and $x\\_j$.”\n\nWe note specifically, that if $c\\_2$ in Eq. 4 is set to zero and the constants on the first and second terms are made very small and their coefficients are identical, this rule is exactly Oja’s.\n\nWe apologize for the lack of clarity in section 2.3.1. We have rewritten it to emphasize the following key points:\nPlasticity rules learned on all synapses outperform rules that only operate on E$\\rightarrow$E synapses, particularly when the rate of synaptic turnover is high. \nPerturbing the learned plasticity rules revealed that the E$\\rightarrow$E was largely the same as in the prior sections.\nWe found E$\\rightarrow$I plasticity heavily depended on a term that was second order in the presynaptic activity, $\\tilde{x}_i x_i$. It always had a positive coefficient.\nDependence on this term implied that the synaptic strength of a neuron’s self inhibition might depend on its activity level, suggesting a type of homeostasis. \n\nThe last paragraph of 2.3.1 now reads:\n\n“To investigate how rules acting on all synapses generated improved time representations, we repeated the dropout analysis introduced in Section 2.1.3, We found that E$\\rightarrow$E plasticity within these learned triples was largely similar to the rules previously learned on E$\\rightarrow$E synapses alone (Eqs. 4 and 6): solutions were consistently sensitive to the removal of $\\tilde{x}\\_i x\\_j$ and $w\\_{ij} \\tilde{x}\\_j x_j$, which appeared consistently with positive and negative coefficients, respectively. Further, dependence on these terms persisted when we trained networks with turnover on E$\\rightarrow$E synapses or I$\\rightarrow$E synapses (Supplement, Fig. 2). As expected, we also found that solutions depended heavily upon terms that acted upon E$\\rightarrow$I and I$\\rightarrow$E synapses. In general, we found that this plasticity was more difficult to interpret due to increased trial to trial variability than the E$\\rightarrow$E plasticity. We did find, however, that the E$\\rightarrow$I plasticity rule consistently depended on the second order presynaptic term $\\tilde{x}\\_i x\\_i$, which always appeared with a positive coefficient, suggesting that E cells project to inhibitory counterparts with a strength that increases (perhaps nonlinearly) with their level of activity. An implication of dependence on this term is that the strength of an E neuron's recurrent inhibition, defined as $|W\\_i^{\\\\rm rec}| = |\\sum\\_k w\\_{i, k}^{\\rm E\\rightarrow I} w\\_{k, i}^{\\rm I\\rightarrow E}|$, where $i$ is the index of the E cell and $k$ indexes the I cells to which it projects, might depend on its level of activity; thus, ablation of excitatory inputs to an E cell might cause its recurrent inhibition to lower in a manner that homeostatically restores its firing.”\n\nWe have additionally fixed references to Figs. 3 and 4.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ilvAGt4dKU", "reviewer": "Reviewer_96SN", "summary": "This paper introduces meta-learned plasticity rules for sequence generating neural circuits. While existing works demonstrated that the dynamics in HVC is generated by both excitatory and inhibitory synaptic updates, their approaches are based on a guessed rule. Motivated from this, the authors experimentally demonstrate that synapses with meta-learned plasticity rules introduce enhanced stability on the network dynamics under noisy environments, which is also biologically plausible (e.g., homeostasis)", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper is overall well written and key question the authors are trying to validate (line 33-34) is presented clearly in the introduction.\n\n- The motivation of this paper to understand the self-organizing networks with meta-learned plastic rules is reasonable and biologically plausible.\n\n- In the field of modeling sequence generating circuits in HVC, the author's investigation is novel considering others are mostly based on a relatively simple and guessed rule.", "weaknesses": "- Though I appreciate the novel problem statement using meta-learned plasticity rules, I could not find technical novelties in the paper. In other words, my primary concern is if this paper presents any novel insights in machine learning perspective. \n\n- For example, meta-learning of plasticity rules has been widely explored in various tasks [1,2,3]. Some of these papers have already employed the similar optimization techniques such as CMA-ES to quickly optimize the meta-learning parameters. Hence, I think the authors should have discussed these methods in the paper. \n\n[1] Miconi, Thomas, Kenneth Stanley, and Jeff Clune. \"Differentiable plasticity: training plastic neural networks with backpropagation.\" International Conference on Machine Learning. PMLR, 2018.\n\n[2] Najarro, Elias, and Sebastian Risi. \"Meta-learning through hebbian plasticity in random networks.\" Advances in Neural Information Processing Systems 33 (2020): 20719-20731.\n\n[3] Rodriguez, Hector Garcia, Qinghai Guo, and Timoleon Moraitis. \"Short-term plasticity neurons learning to learn and forget.\" International Conference on Machine Learning. PMLR, 2022.\n\n- Also, the idea of homeostasis using inhibitory and excitatory synapses are not new in machine learning perspectives [1,2]. I believe the main contribution in this paper is to understand the meta-learned plasticity rules in modeling neuronal sequences. If so, it seems like applying existing methods to understand another problem. Please correct me if I’m wrong.\n\n[1] Yoshida, Naoto, et al. \"Embodiment perspective of reward definition for behavioural homeostasis.\" Deep RL Workshop NeurIPS 2021. 2021.\n\n[2] Kang, Beomseok, Biswadeep Chakraborty, and Saibal Mukhopadhyay. \"Unsupervised 3D Object Learning through Neuron Activity aware Plasticity.\" arXiv preprint arXiv:2302.11622 (2023).\n\n- It would be nice to have a background section for readers who are not familiar with neuronal sequence modeling and zebra finch HVC.\n\n- The current reference (citation) style looks different to NeurIPS format. I think the references should be written in [x] or (x).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments and for the additional references.\n\nWe agree with the reviewer that the novelty of this work primarily lies in the application of an existing technique to a specific question within neuroscience: how might the organization of useful circuit dynamics that accelerate learning be established and maintained without feedback? We argue, however, that the application of this technique to this question is nontrivial. As we argue in the common response, prior studies that employ meta-learning largely operate on feed-forward networks, whose outputs can be rapidly computed, have known solutions within the basis, or operate on a limited basis set (see common response for details and citations). In this work, we show solutions to meta-learning problems can be dense within a provided basis and that coefficient magnitude does not necessarily communicate the contribution of a term to shaping of synapses. We argue this work presents an important case study for other authors seeking to apply meta-learning to questions within neuroscience.\n\nWe thank the reviewer for pointing out important work from Miconi et al., Najarro et al., and Rodriguez, et al. We have added these references.\n\nWe have additionally added a background section on zebra finch physiology, including a review of areas HVC and RA. This section now reads:\n\n“In the zebra finch, premotor nucleus HVC contains excitatory neurons that fire sparsely (typically in one burst of spikes) during song and are purportedly arranged in a feed-forward structure (Fig. 1c). A subset of these cells, known as $\\mathrm{HVC}\\_{\\rm (RA)}$ neurons, project to downstream nucleus RA (robust nucleus of the archistriatum), which in turn projects to vocal neurons of the syrinx and to the brainstem, which regulates respiration \\cite{Hahnloser_2002}. HVC additionally receives excitatory projections from nucleus Uva, which controls the onset of song syllables \\cite{Moll2023} and provides input for the duration of song \\cite{Danish2017}. $\\mathrm{HVC}\\_{\\rm (RA)}$ neurons inhibit each other disynaptically via a population of inhibitory interneurons within HVC \\cite{Kosche_2015}.”\n\nWe agree with the reviewer that the idea of homeostasis via modification of excitatory and inhibitory synapses is not new. However, our work suggests exactly how homeostasis might be maintained via these two distinct forms of input in the sequence generation context. We argue this is valuable because our proposal for homeostatic control can maintain not just the shape of an E cell’s firing rate response but also its timing relative to its input. We clarify this point in Sec. 2.3.3. We additionally thank the reviewer for suggesting additional references we missed.\n\nWe have also fixed the formatting of the references as the reviewer has suggested.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their thoughtful feedback. We appreciate their overall support of the manuscript and their constructive criticism. Below, we address their major concerns.

R1 and R4 noted that the paper lacked a demonstration that the learned plasticity rule accelerated the ability of the self-organized sequential dynamics to support supervised learning of a downstream output pattern. We chose to focus on the self-organization of the timing scaffold as the utility of a timing representation to support motor learning has already been well studied (e.g. Duffy et al. PNAS 2019). Nicola and Clopath (Nat. Commun. 2017) demonstrated that providing a stable representation of time (i.e. a sequential input) accelerates a network’s ability to perform a birdsong-like motor task (see section “High dimensional temporal signals improve FORCE training”). Fiete et al. (J. Neurophys. 2004) argued that temporal sparseness in the premotor drive of HVC vastly simplifies the task of producing some sequential motor output by decomposing it into the approximately independent subtasks of producing the correct motor output at each moment in time; this was also demonstrated in a biologically plausible implementation of RL learning of the same task (Farries and Fairhall J. Neurophys. 2007). Lastly, providing a time input to agents has become popular in the reinforcement learning literature as it permits an agent to adjust its policy in accordance with elapsed time in the task (Wang et al. arXiv preprint arXiv:1611.05763 2016), suggesting another downstream benefit of time representation that deserves further exploration.

Regarding concerns of unclear novelty raised by R2, R3, and R5, we acknowledge this paper builds upon a rich body of work that attempts to evolve Hebbian-like rules via a supervised procedure. We now clarify in the introduction that our unique contribution is the exploration of unsupervised and unrewarded plasticity via meta-learning that organizes and maintains a specific and biologically relevant computational motif, a sequence. While prior work on sequence self-generation used hand-built rules, we learn the optimal plasticity rule by maximizing its ability to represent time within the network dynamics. Further, we believe the inclusion of biological noise, such as instability in synaptic connectivity, in a learning procedure is novel, particularly in the sequence generation context. This question is motivated by recent neuroscience findings about representational drift. Our work illustrates how the inclusion of such noise can shift the optimal learning rule, and demonstrates how plasticity responsible for self-organization can coexist with maintenance mechanisms in a single rule, which to our knowledge is also novel in the context of sequences.

We also point out that this work is novel in its application of meta-learning of plasticity rules to a problem in which the model network is highly recurrent, the rule basis is large, and the optimal rule is not known a priori. Prior work has largely focused on feedforward structures whose outputs can be computed quickly, streamlining the meta-learning process (Confavreux et al. NeurIPS 2020, Shervani-Tabar and Rosenbaum Nat. Commun. 2023, Lindsey and Litwin-Kumar NeurIPS 2020), rule bases that were relatively small (Shervani-Tabar and Rosenbaum Nat. Commun. 2023, Tyulmankov Neuron 2022), or problems where the optimal plasticity rule was already known (Confavreux et al. NeurIPS 2020). Here we show that meta-learning can yield successful rules that are dense in the given basis, requiring us to develop additional techniques to understand their function. Such an example is important to include in the literature for future authors intending to use meta-learning. We also believe the inclusion of an L1-like penalty on synaptic change is novel and represents a more biologically realistic constraint on a plasticity rule than penalizing term coefficients, as other studies have done. 
R1 and R4 also noted section 2.3, “Including inhibitory plasticity,” was difficult to parse. We have rewritten this section to clarify our findings from the rule perturbation analysis and provide more detail regarding the single neuron manipulations we performed (see individual responses).

At the request of R2 and R5, we have added background on zebra finch physiology (see response to R5) that describes the constituent cell types and connectivity within sequence-generating nucleus HVC and the role the nucleus is thought to play in song production.
Regarding broader impact, our work suggests a specific role for plasticity that is unmodulated by feedback: the setup of simple computational motifs. We hypothesize that the plasticity that organizes these computational motifs might be extremely specific to the dynamics being organized and may not require supervision. We feel that this work will inspire future work to seek plasticity rules that underlie the self-organization of other common computational motifs, such as line and ring attractors. From the standpoint of experimental predictions, this work provides a hypothesis for why behaviors like path integration may be innate to some systems. It additionally suggests that the establishment of such simple computational motifs in the brain may not require dopaminergic or other reward-related inputs.

Meta-review: This paper proposes a procedure for discovering the synaptic plasticity rules by which a neural network can learn sequences. There were some questions as to the novelty and generalizability of this work; however, there was broad agreement that this work represents an interesting contribution to the field of theoretical neuroscience. Therefore, I recommend acceptance.
Decision: Accept (poster)