Model Based Inference of Synaptic Plasticity Rules
Authors: Yash Mehta
Danil Tyulmankov
Adithya E. Rajagopalan
Glenn C Turner
James E Fitzgerald
Jan Funke
Venue: NeurIPS 2024 poster (2024)
Keywords: computational neuroscience
plasticity rules
synaptic plasticity
biologically plausible learning
Primary area: neuroscience_and_cognitive_science
TL;DR: We developed a computational method to infer complex synaptic plasticity rules from neural and behavioral data, revealing new insights like active forgetting in reward learning in flies.

Abstract: Inferring the synaptic plasticity rules that govern learning in the brain is a key challenge in neuroscience. We present a novel computational method to infer these rules from experimental data, applicable to both neural and behavioral data. Our approach approximates plasticity rules using a parameterized function, employing either truncated Taylor series for theoretical interpretability or multilayer perceptrons. These plasticity parameters are optimized via gradient descent over entire trajectories to align closely with observed neural activity or behavioral learning dynamics. This method can uncover complex rules that induce long nonlinear time dependencies, particularly involving factors like postsynaptic activity and current synaptic weights. We validate our approach through simulations, successfully recovering established rules such as Oja's, as well as more intricate plasticity rules with reward-modulated terms. We assess the robustness of our technique to noise and apply it to behavioral data from \textit{Drosophila} in a probabilistic reward-learning experiment. Notably, our findings reveal an active forgetting component in reward learning in flies, improving predictive accuracy over previous models. This modeling framework offers a promising new avenue for elucidating the computational principles of synaptic plasticity and learning in the brain.

Model-based inference of synaptic plasticity rules
Yash Mehtal.2
Danil Tyulmankoy311
Adithya E. Rajagopalan 1.5
Glenn C. Turner
James E. Fitzgerald
Jan Funke 1*
JJanelia Research Campus, Howard Hughes Medical Institute
Department of Cognitive Science, Johns Hopkins University
Center for Theoretical Neuroscience, Columbia University
Viterbi School of Engineering, University of Southern California
Center for Neural Science, New York University
Departmentt of Neurobiology, Northwestern University
Abstract
Inferring the synaptic plasticity rules that govern learning in the brain is a key chal- lenge in neuroscience. We present novel computational method to infer these rules from experimental data, applicable to both neural and behavioral data. Our approach approximates plasticity rules using parameterized function, employing either truncated Taylor series for theoretical interpretability multilayer percep- trons. These plasticity parameters are optimized via gradient descent over entire trajectories to align closely with observed neural activity behavioral learning dynamics. This method can uncover complex rules that induce long nonlinear time dependencies, particularly involving factors like postsynaptic activity and current synaptic weights. We validate our approach through simulations, successfully re- covering established rules such Oja's 's well with
intricate plasticity rules terms. We assess the robustness of our technique to noise and apply it to behavioral data from Drosophila in a probabilistic reward-learning experiment. Notably, our findings reveal an active forgetting component in reward learning flies, improving predictive accuracy over previous models. This mod- eling framework offers a promising new avenue for elucidating the computational principles of synaptic plasticity and learning in the brain.
Introduction
Synaptic plasticity, the ability of synapses to change their strength, key neural mechanism un- derlying learning and memory in the brain. These synaptic updates are driven by neuronal activity, and they in turn modify the dynamics of neural circuits. Advances in neuroscience have enabled the recording of neuronal activity et al., 2018; Zhang et al., 2023), and connectome data for various organisms becoming increas- ingly available (Winding al., 2023; Takemura et al., 2023; Hildebrand et al., 2017; Scheffer al., 2020). However, the inaccessibility of direct large-scale recordings of synaptic dynamics leaves the identification of biological learning rules an open challenge. Existing neuroscience literature (Citri& & Malenka, 2008; Morrison et al., 2008) suggests that synaptic changes are functions of lo- cal variables such presynaptic activity, postsynaptic activity, and current synaptic weight, well as a global reward signal. Uncovering the specific form of this function in different brain circuits promises profound biological insights and holds practical significance for developing more biologi- cally plausible learning algorithms for AI, particularly with neuromorphic implementations (Zenke & Neftci, 2021).
unprecedented scale (Steinmetz al., 2018; Vanwalleghem
Joint senior authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Model Neural Network
m
Neural Activity Trace
Behavior Trace
m
Infering
Lpcu(
Learning Rule
Behavior m((
Stimulus x(t)
Aufj - salay "i, why p9))
(-1)vcC(0)
mm
ma
Cassa (
Time
Data
Figure I: Schematic overview the proposed method. Animal-derived time-series data (yellow) and a
in silico model (blue) generate trajectories (*)) and m() loss function quantifies trajectory mismatch to produce gradient, enabling the inference of the synaptic plasticity rule go.
In this paper, we introduce gradient-based method for inferring synaptic plasticity rules. Our method optimizes parameterized plasticity rules to align with either neural and behavioral data, thereby elucidating the mechanisms governing synaptic changes biological systems. We utilize interpretable models of plasticity, allowing direct comparisons with existing biological theories and addressing specific questions, such as the role weight decay in synaptic plasticity or postsynaptic dependence, We validate approach for recovering plasticity rules using synthetic neural activity or behavior2. Finally, applying our model to behavioral data from fruit flies, we uncover an active forgetting mechanism in the neural circuitry underlying decision making. This readily adaptable modeling framework offers new opportunities for exploring the core mechanisms behind learning and memory processes in variety of experimental paradigms.
2 Method overview
Our goal is to infer synaptic plasticity function by examining neural activity or behavioral tra- jectories from animal learning about its environment. Specifically, we aim to find function that prescribes changes in synaptic weights based on relevant biological variables. For simplicity, we consider model with plasticity localized to single layer of neural network:
sigmoid (w())((())
where the vector (2) represents the input the plastic layer (Figure 1, "stimulus") and (() is the resulting postsynaptic neuron activity time The synaptic weight matrix W is updated at each time step based on a parameterized biologically plausible plasticity function go- The change in synaptic weight between neurons and j is given by
(1) Aw = go
1j r(())
x
U
w
where 0 are the (trainable) parameters of the function, 2 is the presynaptic neural activity, y/ the postsynaptic activity, w is the current synaptic weight between neurons and j. and (()) is global reward signal that influences all synaptic connections. However, it may be the case that we do not have direct access to the neuronal firing rates (t),, We therefore further define a (fixed) readout function f that determines the observable variables (1) of the network, given by
((y
(3)

---

A
B
Ami
D.
of Mi
Weight
Error
Ground Truth (Known Rulle)
Neural
Training Epochs
100
8,r
Activity
200
4b-3
dade Value
0.3
30
0.0
Model (Leamed Rule)
i
Neural Activity
300
2w-
-0.5
163
6,0
400
-1.0
Auny Pegler
100
200
Training Epochs
D
Sparseness
E
1.0
F
G
1.0
0.0 05
0.8
Noise Scale
score
R score
Buto Volue
0.6
0.6
0.0
0.6
0.4
0.4
0.8
0.2
0.2
-0.5
-1.0
0.5
6.0
0.8
0.0
0.9
0.7 Sparseness
100
200
12
Noise Scale
Training Epochs
Figure 2: Recovery of Oja's plasticity rule from simulated neural activity. (A) Schematic of the models used to simulate neural activity and infer plasticity. (B) Mean-squared difference between ground-truth and model synaptic weight trajectories over time (horizontal axis) over the course of training epochs (vertical axis). (C) The evolution of 9 during training. Coefficients 0110 and fu21, corresponding to Oja's s rule values (1) are highlighted in orange.
(D) R scores over weights, under varying noise and sparsity conditions in neural data. (E, F) Boxplots of distributions, across 50 seeds, corresponding to the first column (E) and row in (D). (G) The evolution of learning rule coefficients over the course of training showing inaccurate 0 recovery under high noise and sparsity conditions.
In the context of neural activity fitting, the readout is a subset of (*),, whereas for behavioral models the readout aggregates ( to yield the probability of a specific action. We introduce our specific choices for readout functions in the following sections.
We use stochastic gradient descent (Kingma & Ba, 2014) to optimize the parameters 0 of the plas- ticity rule go- At each iteration, use the model (Equation 1-3) to generate a length-T trajectory m1) ....,
.....,m()) (2)) (Figure 1, blue traces), driven by input stimuli (1) ..... xx') (Figure 1, black box). We then use backpropagation through time to compute the gradient of the loss (Figure 1, purple) between the model trajectory and the corresponding experimental observations (1)) (!)),...,o" (T) (Fig- ure 1, orange) generated using the same input stimulus:
T
(())
T
Σ
(m (0)))
where the choice of & depends on the particular modeling scenario. specified in the following sec- tions. practice, Equation may also be summed over multiple trajectories generate mini- batch.
3 Inferring plasticity rule from neural activity
To validate our approach on neural activity, we generate synthetic neural trajectories of observed outputs (t) from single-layer feedforward network that undergoes synaptic plasticity according to the well-known Oja's rule (Oja, 1982). At each timestep, the weight updates depend on pre- and post-synaptic neuronal activity, well the strength of the synapse itself (Figure 2A, top),
Away T334
Vi Waj.
(5)
3

---

where we omit the time index t for brevity (see subsection A.3 for details). To infer the plasticity rule. we use model network with an architecture identical to the ground-truth network (Figure 2A, bottom). While the exact circuit architecture may not always be known in biological data (par- ticularly when studying mammalian brains), this assumption was made to show that our approach accurately infers plasticity rules in a scenario where generative and predictive circuit architectures were matched. Increasingly available connectomic information in biological systems can be used to design sufficiently accurate model architectures for our approach to work, as we will show later on in this paper (Bentley et al., 2016; Hildebrand et al., 2017: Scheffer et al., 2020). However, is to be noted that connectomes can miss important information relevant to plasticity and may cause mismatches between model and generative architectures that could lead the model to incorrect out- comes (Liang & Brinkman, 2024). This should be kept in mind when interpreting the plasticity rules estimated by our approach.
Following previous work (Confavreux et al., 2020), we parameterize the model's plasticity function with a truncated Taylor series,
2
Taylor
Σ
24 w/
(6)
0,0,y=0
where the coefficients Oagy are learned. Note that Oja's rule be represented within this family of plasticity rules by setting 0110 = 1, 0021 = 11, and all others to zero. Finally, we compute the loss the mean squared error (MSE) between the neural trajectories produced by the ground truth
network and the model:
( (t) (()) ||o( mm 11
where we let m() = -)(),, assuming all neurons in the circuit are recorded (but see next section for analysis of sparse recordings).
3.1 Recovering Oja's rule
Despite the fact that the model is optimized using neuronal trajectories, the error of the synap- tic weight trajectories decreases over the course of training (Figure 2B), indicating that the model successfully learns to approximate the ground-truth plasticity rule. More explicitly examining the coefficients 02p, over the course of training illustrates the recovery of Oja's rule 0110 and 0021 approach 1 and 1, respectively, and all others go to zero (Figure 2C).
To evaluate the robustness of our method, we assess how both noise and sparsity affect the model's performance (Figure 2D). We first consider the case where all neurons in the circuit are recorded, and we vary the degree of additive Gaussian noise in the recorded neurons. We find that the model's s performance (R² score between the ground-truth and model synaptic weight trajectories calculated on separate held-out test set) decreases with increasing noise variance (Figure 2E). To simulate (t)
varying sparsity levels, we consider the readout m (t) = ()) = (yk ....................... (f) ) to be the activ- ity of random subset of all N postsynaptic neurons ((t), and we use the corresponding subset ((0) = (o .....' o of recorded neurons from the ground-trute network in Equation to optimize the plasticity rule. Our model maintains high level of accuracy even when data is available from only 50% of the neurons (Figure 2F). This resilience to sparsity and noise is beneficial given that experimental neural recordings often suffer from these issues. However, we note that the model struggles to learn a sparse set of parameters for the plasticity rule when faced with both high record- ing sparsity and noise. The evolution of the plasticity parameters during training in this case is illustrated in Figure 2G. Together, these results show that our approach can accurately infer learning rules from neural trajectories in wide array of recording conditions.
4 Inferring plasticity rules from behavior
Our approach can also be applied to behavioral data. This particularly important because behav- ioral experiments are more widely available and easier to conduct than those that directly measure neural activity. We first validate the method on simulated behavior, mimicking decision-making experiments in which animals are presented with series of stimuli that they choose to accept or reject. The animals' choices result in rewards and subsequent synaptic changes at behaviorally rel- evant synapses.

---

A
B
Dall =ff BLRO)
Thee We gPa
Ground Truth
Choice
Sympttceeeeghss
(Knowin
Behavior
score
y
0.5
0.0
Model (Leamed Rule)
Choice
Behavior
-1.5
Awy
50
100
150
200
250
Trial
Taylor
MLP
D
100
% Dev Exclained
-Value
VVllue
0.5
0.5
Ban
50
100
252
330
Epocns
Plsticty
Taylor hip
Figure 3: Recovery of a reward-based plasticity rule from simulated behavior. (A) Schematic of the models used to stimulate behavior and infer plasticity rules. (B) The evolution of the weight of a single synapse, trained with 9p
Taylor and go MLP, compared against weight from a known reward- based update rule. (C) R²2 distributions on weights across seeds, corresponding to varied weight initializations and stimulus encodings. (D) The evolution of 0 during training, with 0 10- corresponding to ground truth rule (value 1), highlighted red, (E) Distribution of final inferred 9 values across seeds, showing accurate identification of the relevant term from the ground truth learning rule. (F) The goodness of fit between ground truth behavior and model predictions plotted as the percent deviance explained.
For this proof-of -rrinccppee our ground-truth network architecture (Figure 3A, top) is inspired by recent studies that have successfully mapped observed behaviors to plasticity rules in the mushroom body (MB), the learning and memory center of the fruit fly Drosephila melanogaster (Li et al., 2020; Modi et al., 2020; Davis, 2023). Our neural network's 's three layer structure mimics the MB's neural architecture (see subsection A.4 for details). The readout (t) a series of binary decisions to either "accept" "reject" the presented stimulus based on the average activity of the output layer. A probabilistic binary reward R E {0, 1} is provided based on the choice. The reward signal is common to all synapses, which could be interpreted like dopamine. This reward leads to changes in the plastic weights of the network, determined by the underlying synaptic plasticity rule.
global neuromodulatory signal
Plasticity occurs exclusively between the input and output layers. We simulate a covariance-base learning rule (Loewenstein & Seung, 2006) known from previous experiments (Rajagopalan et al., 2023). The change in synaptic weight Aw;y is determined by the presynaptic input æj. and a global reward signal This reward signal is the deviation of the actual reward R from its expected value E[R], which we calculate as a moving average over the last 10 trials. We neglect hypothetical dependencies on yı because they are known not impact reward learning in the fly mushroom body (although see also Table and Appendix Table 3 for experiments with alternative plasticity rules):
Awyy = (R - E(R)).
We model a plastic layer of neural connections that gives rise to learned behavior. The synaptic weights of the model are initialized randomly, reflecting the fact that the initial synaptic config- urations are usually unknown priori in real-world biological systems. We consider a plasticity function parameterized through either a truncated Taylor series or multilayer perceptron (MLP):
2
9o
Taylor
Oadrsa Vi w7yr
MLP MLPa(1; Wr, Wig,
(9)
a,8,7,8=00
5

---

Table I: Assessment of various reward-based plasticity rules: R² scores for weight and individual neural activity trajectories, and the percentage of deviance explained for behavior. Refer to Ap- pendix Table for a comprehensive list simulated plasticity rules.
MLP
Taylor
Plasticity Rule Awiy
R²
R² Activity
% Deviance
R² Weights
R²
% Deviance
Weights
Activity
FyT
0.85
0.96
64.76
0.78
0.94
61.91
r 0.05g:
0,97
0,97
34.55
0,96
0.97
34.36
3 r
0.05mr
0.87
0.91
57.01
0.70
0.86
51.80
Zzr" 0.05x, w.
0.85
0.96
53.04
0.78
0.92
51.30
J. J 0.05r
0.27
0.34
79.92
0.41
0.51
84.22
where the ground truth reward r is used as the reward value in the weight update function. We use Binary Cross-Entropy (BCE) as the loss function, which is proportional to the model's negative log-likelihood function, to quantify the difference between the observed decisions and the model's predicted probabilities of accepting stimulus.
BBE (m (1) ())(())) (C) (og(m(()) (1-o(() log(1 - m ()).
(10)
Crucially, the training data only consists of these binary decisions (accept reject), without direct access to the underlying synaptic weights or neural activity.
4.1 Recovering reward- plasticity from behavior
Figure 3B presents the weight dynamics of three networks: the ground-truth synaptic update mech- anism, as well as those fitted with MLP Taylor series. Both the ground-trutt network and our model use architecture with 2 input neurons, 10 neurons in the hidden layer, and I output neuron with a trajectory length of 240 time steps (see Appendix subsection A.4). Our evaluation metries high R²2 values for both synaptic weights and neural activity afffimm the robustness of our models in capturing the observed data (Figure 3C). The method accurately discerns the plasticity coefficient of the ground truth rule (Figure 3D,E), albeit with reduced magnitude. The model also does good job at explaining the observed behavior (Figure 3F), where we use the percent deviance explained (see Appendix subsection A.4) as the performance metric.
We also consider alternative plasticity rules in the ground-trutt -ruth network. Table summarizes the recoverability of various reward- based plasticity rules for both MLP and Taylor series frameworks, with results averaged over random seeds. Note that solely reward-based rules (without [[ or w) are strictly potentiating, as they lack the capacity for bidirectional plasticity. This unidirectional potentiation ultimately results in the saturation of the sigmoidal non-linearity. Therefore, it is pos- sible to simultaneously observe high R² values for neural activities with low Rº values for weight trajectories.
We further investigate the scalability of our method by varying the length of the observed trajectory and the number of neurons in the hidden layer (see Table 2). The model's goodness-of-f generally improved with longer simulations, likely due to more data points for inferring the plasticity rule. However, R² values for activity and weights peaked before declining, suggesting potential overfit- ting on very long trajectories. Model performance remained consistent when scaling to larger hidden layers, assuming the same plasticity rule shared by all synapses.
Application: inferring plasticity in the fruit fly
In extending our results to biological data, we explore its applicability to the decision decision-making be- havior in Drosophila melanoguster that inspired our simulated behavior results. Recent research (Rajagopalan et al., 2023) employed logistic regression to infer learning rules governing synaptic plasticity in the mushroom body, identifying a rule that incorporates the difference between re- ceived and expected reward information when modulating synaptic plasticity. However, logistic

---

Table 2: Scalability analysis with respect to trajectory length (with a hidden layer size of 10) and hidden layer size (with trajectory length of 240), assuming the ground-truth learning rule of Amy = Zyr- = zj (R - E(R] and using the Taylor series parameterization. Results are aver- aged over three runs with different random seeds.
Trajectory Length
Hidden Layer Size
30
60
120
240
480
960
1920
10
50
100
500
1000
Ha Weights
0.79
0.74
0.70
0.78
0.72
0.53
0.64
0.78
0.75
0.79
0.79
0.79
R²
Activity
0.92
0.91
0.91
0,94
0.95
0.87
0.92
0.94
0.94
0.95
0.95
0.95
% Deviance
39.52
40.14
48.06
61.91
76.90
73.78
79.66
61.91
62.29
62.25
62.27
62.26
regression cannot be used to infer plasticity rules that incorporate recurrent temporal dependencies, such as those that depend on current synaptic weights. Our method offers more general approach. Specifically, we apply our model behavioral data obtained from flies engaged in two-alternative choice task, as outlined in Figure 4AA. This allows us to investigate two key questions concerning the influence of synaptic weight on the plasticity rules governing the mushroom body.
5.1 Experimental setup and details
In the experimental setup, individual flies are placed in symmetrical Y-arena where they are pre- sented with a choice between two odor cues. Each trial starts with the fly in an arm filled with clean air (Fig. 4A, left). The remaining two arms are randomly filled with two different odors, and the fly was free to navigate between the three arms. When the fly enters the 'reward zone' at the end of an odorized arm, choice was considered to have been made (Fig. 4A, right). Rewards are then dispensed probabilistically, based on the odor chosen. For model fitting. we use data from 18 flies, each subjected to protocol that mirrors the trial and block structures in the simulated experiments presented previously. Over time, flies consistently showed a preference for the odor associated with higher probability of reward, and this preference adapted to changes in the relative value of the options (Fig. 4B; example fly (Rajagopalan et al., 2023)).
5.2 Plasticity in the fruit fly includes a synaptic weight decay
Existing behavioral studies in fruit flies have shown that these insects can forget learned associations between stimuli and rewards over time (Shuai et al., 2015; Aso & Rubin, 2016; Berry et al., 2018; Gkanias et al., 2022). One prevailing hypothesis attributes this forgetting to homeostatic adjustments in synaptic strength within the mushroom body (Davis & Zhong, 2017; Davis, 2023). However, earlier statistical approaches aimed estimating the underlying synaptic plasticity rule present in the mushroom body were unable to account for recurrent dependencies such as synapse strength (Rajagopalan et al., 2023). Here we explore two types of plasticity rules: one based solely on reward and presynaptic activity, and another that incorporates a term dependent on current synaptic weight- Way (0oo1). Both rule types allocate significant positive weights term representing the product of presynaptic activity and reward (Fig. 4C, gray). Our results indicate that the model with a weight-dependent term offers a better fit to observed fly behavior (Wilcoxon signed-rank test: p = 5 x 10 10-5, 5, Fig. 4D), whereas the model without it matched the performance reported in Rajagopalan al. (2023). Intriguingly, our analysis additionally reveals that the inferred learning rule assigns negative value to the weight-dependen term (Fig. 4C). This finding also held in validation experiment that assessed how well the inferred plasticity rule generalizes to unseen biological data (Appendix subsection A.6). This negative sign aligns with the hypothesis that a weight-dependen decay mechanism operates at these synapses. The relative-magnitude of this decay term compared to the positive learning-related terms suggests that forgetting happens over a slightly longer-time scale than learning, agreement with observed time-scales forgetting in behavioral experiments Shuai et al. (2015); Davis & Zhong (2017).
7

---

A
B
Odor
Odor
Trial
Cholice Ratio
Reward Ratio u Unrewarded Rewarded
80
20
89
11
Reward
Choice Ratio
100:0
Optogenetic
Zone
Sugar Reward
5cm
Exhaust
Trial
50:50
0:100
Air
80
160
240
Trial Number
C
1
Plasticlyy Rule lAin)
Without teem
Se-03
Se-05
% Deviance Explained & wlooo
x, torn
% Deviance Explained
Rajopopalan
Bu Value
model
-2
-0.5
-4
f0o0
80ol
0010
0100
0110
Rajagopalan Without w
With Waj
Flies
E
Hasticity Rule (hx.))
expected reward
p-value:
% Deviance Explained
% Deviance Explained
14
0.067
0,p Value
12
10-
-1
-0.5-
-2
90oo 0001
6010 600
0110
R-E(R)
R
Flies
Figure 4: Inferring principles of plasticity in the fruit fly. (A) Schematic of the experimental setup used to study two-alternative choice behavior in flies. Left Design of arena showing odor entry ports and location of the reward zones. Right Description of the trial structure, showing two example trials. (B) The behavior of an example fly in the task. Top Schematics indicate the reward baiting probabilities for each odor in the three blocks. Bottom Individual odor choices denoted by rasters, tall rasters rewarded choices, short rasters unrewarded choices. Curves show 10-trial averaged choice (red) and reward (black) ratios, and horizontal lines the corresponding averages over the 80-trial blocks. (C) Final inferred 0 value distribution across 18 flies, comparing models with and without way term and the method from Rajagopalan al. (2023). Plasticity rule terms are as follows: bias (0oon), Wij (0601). ). Ij (01on), (0010), I,p (0 (D) Left Goodness of fit between fly behavior and model predictions plotted as the percent deviance explained (n 18 flies). Right Change in the percent deviance explained calculated by subtracting percent deviance explained of model without Way (0001) term from that of model with wiy (f001) term. (E.F) Same as (C,D), except comparing models that do or don't It incorporated reward expectation. Since these models include weight dependence, they cannot be fit using Rajagopalan et (2023)'s method.
5.3 Incorporating reward expectation provides better fit than reward alone
Rajagopalan and colleagues used reward expectations (defined the average reward received over the last approximately 'n' trials see Appendix A.4.2) to generate bidirectional synaptic plasticity. Our discovery of negative weight-dependent component the plasticity rule provides an alternate mechanism for bidirectional plasticity, raising the question of whether the neural circuit really needs to calculate reward expectation. Could a plasticity rule incorporating the product of presynaptic activity and absolute reward combine with weight-dependent homeostatic term to approximate

---

plasticity rule that involves reward expectation? To answer this, we contrast two models: one using only the absolute reward and another using reward adjusted by its expectation, both complemented by weight-dependent terms. Our analyses show that adding a weight-dependent term enhances the predictive power of both models (Fig 4F,F). However, the model that also factors in reward expectations provides superior fit for the majority of flies in the data set (Wilcoxon signed-rank test: = 0.067: Fig 4F). These compelling preliminary findings reaffirm the utility of reward expectations for fly learning, and larger behavioral datasets could increase the statistical significance of the trend. Overall, our model-based inference approach, when applied to fly choice behavior, suggests that synaptic plasticity rules in the mushroom body of fruit flies are more intricate than previously understood. These insights could potentially inspire further experimental work to confirm the roles of weight-dependen homeostatic plasticity and reward expectation in shaping learning rules.
Related work
Recent work has begun to address the question of understanding computational principles governing synaptic plasticity by developing data-driven frameworks infer underlying plasticity rules from neural recordings. Lim et al. (2015) infer plasticity rules, as well as the neuronal transfer function, from firing rate distributions before and after repeated viewing of stimuli in a familiarity task. The authors make assumptions about the distribution of firing rates, as well as a first-order functional form of the learning rule. Chen et al. (2023) elaborate on this approach, fitting plasticity rule by either Gaussian process or Taylor expansion, either directly to the synaptic weight updates or indirectly through neural activity over the course of learning. Both approaches consider only the difference in synaptic weights before and after learning. In contrast, our approach fits neural firing rate trajectories over the course of learning and can be adapted to fit any parameterized plasticity rule.
Other work infers learning rules based on behavior instead. Ashwood al. (2020) uses a Bayesian framework to fit parameters of learning rules in a rodent decision decision-making task. The authors ex- plicitly optimize the weight trajectory in addition to parameters of the learning rules, requiring an approximation to the posterior of the weights. Our approach directly optimizes the match between the model and either neural or behavioral data, as defined by a pre-determined loss function. In- terestingly, despite this indirect optimization, we see matching in the weight trajectories as well. Rajagopalan al. (2023) fit plasticity rules in the same fly decision makinn task we consider here. They assumed that the learning rule depended only on presynaptic activity and reward, which recasts the problem as logistic regression and permits easy optimization. Our approach allows us to account for arbitrary dependencies, such as on postsynaptic activities and synaptic weight values, and we thereby identify weight decay term that leads to active forgetting.
Ramesh et al. (2023) also consider optimization of plasticity rules based on neural trajectories. Unlike our approach which uses an explicit loss function, the authors use generative adversarial (GAN) approach to construct a generator network, endowed with a plasticity rule, to produce neu- ral trajectories that are indistinguishable by a discriminator network from ground-truth trajectories. Although, in principle, this approach can account for arbitrary and unknown noise distributions, it comes at cost of high compute resources, a need for large amounts of data, and potential training instability - all well-known limitations of GANs. In practice, it is common to make an assumption about the noise model through an appropriately defined loss function (e.g. Gaussian noise for the MSE loss we use here). Importantly, the authors note degeneracy of plasticity rules- different rules leading to similar neural dynamics. We see similar results, although we interpret this as "slop- piness" (Gutenkunst et al., 2007) overparameterized models being underconstrained by the data (e.g. functions z and x² are indistinguishable if the only values of x which are sampled are 0 and 1). We hypothesize that in the infinite data limit the fitted plasticity rules would, in fact, be unique.
Previous work has also considered inferring plasticity rules directly from spiking data (Stevenson & Koerding, 2011; Robinson et al., 2016; Linderman et al., 2014; Wei & Stevenson, 2021) or selecting families of plausible rules in spiking neural network models (Confavreux et al., 2024). Due to the gradient-based nature of our optimization technique, our proposed approach can account for such data by converting spike trains to a rate-based representation by smoothing. Alternatively, black- box optimization techniques such as evolutionary algorithms can be used to circumvent the need

---

for computing gradients, allowing non-differentiable plasticity rules like spike-timing dependent plasticity be used as model candidates.
Alternatively, meta-learning techniques (Thrun & Pratt, 2012) can be used to discover synaptic plasticity rules optimized for specific computational tasks (Tyulmankov al., 2022: Najarro & Risi, 2020; Confavreux et al., 2020; Bengio et al., 1990). The plasticity rules are represented as parameterized functions of pre- and post-synaptic activity and optimized through gradient descent or evolutionary algorithms to produce a desired network output. However, the task may not be well- defined in biological scenarios, and the network's computation may not be known a priori. Our method obviates the need for specifying the task, directly inferring plasticity rules from recorded neural activity or behavioral trajectories.
Finally, Nayebi et al. (2020) do not fit parameters of a learning rule at all, but use a classifier to distinguish among four classes of learning rules based various statistics (e.g. mean, variance) of network observables (e.g. activities, weights). Similarly, Portes et al. (2022) propose a metric for distinguishing between supervised and reinforcement learning algorithms based on changes in neural activity flow fields in a recurrent neural network.
7 Limitations and future work
Despite its strengths, our model has several limitations that offer avenues for future research. One such limitation is the lack of complex temporal dependencies in synaptic plasticity, neglecting bio- logical phenomena like metaplasticity (Abraham, 2008). Extending our model to account for such temporal dynamics would increase its biological fidelity. Another issue is the model's "sloppiness" the solution space; it can fail to identify a unique, sparse solution even with extensive data. As neural recording technologies like Neuropixels (Steinmetz et al., 2021, 2018) and whole-brain imag- ing (Vanwalleghem et al., 2018) become more advanced, and connectome data for various organisms become increasingly available (Bentley et al., 2016; Hildebrand et al., 2017: Schefferr et al., 2020; Winding et al., 2023), there are exciting opportunities for validating and refining our approach. In- corporating these high-resolution, large-scale datasets into model is crucial next step. In partic- ular, future work could focus on scaling our approach to work with large-scale neural recordings and connectomics, offering insights into the spatial organization of plasticity mechanisms. Such refine- ments will be important when applying our approach to larger and more densely connected brains, such mammalian ones. Additional considerations for future research include the challenges posed by unknown initial synaptic weights, the potential necessity for exact connectome information, and the adequacy of available behavioral data for model fitting.
10

---

References
Wickliffe Abraham. Metaplasticity: tuning synapses and networks for plasticity. Nature Reviews Neuroscience, 95).387-387 2008.
Zoe Ashwood, Nicholas Roy, Ji Hyun Bak, and Jonathan W Pillow. Inferring learning rules from animal decision-making Advances in Neural Information Processing Systems, 33:3442-3453, 2020.
Yoshinori Aso and Gerald M Rubin. Dopaminergic neurons write and update memories with cell- type-specific rules. Elife, 5:e16135, 2016.
Yoshua Bengio, Samy Bengio, and Jocelyn Cloutier. Learning a synaptic learning rule. Cite- seer, 1990. URL https://citeseerx. ist. http:/oooooonttppddrreeeetteeppp db--4b00dssfaaa00000ff0000006660
Barry Bentley, Robyn Branicky, Christopher I. Barnes, Yee Lian Chew, Eviatar Yemini, Edward T Bullmore, Petra E Vértes, and William R Schafer. The multilayer connectome of cacnorhabditis elegans. PLoS computational biology, 22112)e100528) 2016.
Jacob A Berry, Anna Phan, and Ronald L Davis. Dopamine neurons mediate learning and forgetting 2
through bidirectional modulation of a memory trace. Cell Rep., 25(3):651-662.e) October 2018.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Min Milne,, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http: //github. www/coo//////eeeee google/oaaaa.
Shirui Chen, Qixin Yang. and Sukbin Lim, Efficient inference of synaptic plasticity rule with gaus- sian process regression. /science, 26(3), 2023.
Ami Citri and Robert C Malenka. Synaptic plasticity: multiple forms, functions, and mechanisms. Neumosychopharrmacoogy 33(1):18-41, January 2008.
Basile Confavreux, Friedemann Zenke, Everton Agnes, Timothy Lillicrap, and Tim Vogels. A meta-learning approach to (re) discover plasticity rules that carve a desired function into a neural network. Advances Neural Information Processing Systems, 18:16398-16408 2020.
Basile Confavreux, Poomnia Ramesh, Pedro J Goncalves, Jakob H Macke, and Tim Vogels. Meta- learning families of plasticity rules in recurrent spiking networks using simulation- based infer- ence. Advances in Neural Information Processing Systems, 36, 2024.
Ronald I. Davis. Learning and memory using drosophila melanogaster: a focus on advances made in the fifth decade of research. Genetics, 224(4), August 2023.
Ronald L Davis and Yi Zhong. The biology of Forgetting-A A perspective. Neuron, 95(3):490-503, August 2017.
Evripidis Gkanias, Li Yan McCurdy, Michael N Nitabach, and Barbara Webb. An incentive circuit for memory dynamics in the mushroom body of drosophila melanogaster. Elife, 11, April 2022.
Ryan N Gutenkunst, Joshua J Waterfall, Fergal Casey, Kevin S Brown, Christopher R Myers, and James P Sethna. Universally sloppy parameter sensitivities in systems biology models. PLoS Computational Biology, (100)e189, October 2007. ISSN 1553-7358. doi: 10. 13.7///uunnal.pchi. 0030189. URL http::/tttee //xx doi www.ddii.....o....... wwgpo../........o/o 1371/journal. 0030088..
David Grant Colburn Hildebrand, Marcelo Cicconet, Russel Miguel Torres, Woohyuk Choi, Tran Minh Quan, Jungmin Moon, Arthur Willis Wetzel, Andrew Scott Champion, Brett Jesse Graham, Owen Randlett, et al. Whole-brai serial-section electron microscopy in larval zebrafish. Nature, 545(7654):345-3444 2017.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, arXiv preprint arXiv.1412.6980, 2014.

---

Feng Li, Jack W Lindsey, Elizabeth C Marin, Nils Otto, Marisa Dreher, Georgia Dempsey, lldiko Stark, Alexander S Bates, Markus William Pleijzier, Philipp Schlegel, Aljoscha Nem, Shin- Ya Takemura, Nils Eckstein, Tansy Yang, Audrey Francis, Amalia Braun, Ruchi Parekh, Marta Costa, Louis K Scheffer, Yoshinori Aso, Gregory Sxe Jefferis, Larry F Abbott, Ashok Litwin-Kumar, Scott Waddell, and Gerald M Rubin. The connectome of the adult drosophila mushroom body provides insights into function. Elife, 9, December 2020.
Tong Liang and Braden A W Brinkman. Statistically inferred neuronal connections in subsampled neural networks strongly correlate with spike train covariances. Phys Rev E, 109(4-1):04404, April 2024.
Sukbin Lim, Jillian McKee, Luke Woloszyn, Yali Amit, David Freedman, David L Sheinberg, and Nicolas Brunel. Inferring learning rules from distributions of firing rates in cortical neurons. Nature neuroscience, 18(12)118441110, 2015.
Scott Linderman, Christopher H Stock, and Ryan P Adams. A framework for studying synaptic plasticity with neural spike train data. Advances in neural information processing systems, 27, 2014.
Yonatan Loewenstein and H Sebastian Seung. Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proceedings of the National 2006.
Academy of Sciences,
Mehrab N Modi, Yichun Shuai, and Glenn C Turner. The drosophila mushroom body: From archi- tecture algorithm in a learning circuit. Annu. Rev. Neurosci., 43:465-484, July 2020.
Abigail Morrison, Markus Diesmann, and Wulfram Gerstner. Phenomenological models of synaptic plasticity based on spike timing. Biological cybernetics, 98:459-478, 2008.
Elias Najarro and Sebastian Risi. Meta-leaming through hebbian plasticity in random networks. Advances in Neural Information Processing Systems, 11-20719-20735 2020.
Aran Nayebi, Sanjana Srivastava, Surya Ganguli, and Daniel I. Yamins. Identifying learning rules from neural network observables. Advances in Neural Information Processing Systems, 33:2639- 2650, 2020.
Erkki Oja. Simplified neuron model as principal component analyzer. Journal of mathematical biology, 15:267-273, 1982.
Jacob Portes, Christian Schmid, and James M Murray. Distinguishing learning rules with brain machine interfaces. Advances in neural information processing systems, 35:25937-25950, 2022.
Adithya E Rajagopalan, Ran Darshan, Karen L Hibbard, James E Fitzgerald, and Glenn C Turner. Reward expectations direct learning and drive operant matching in drosophila. Proceedings of the National Academy of Sciences, 120639):e22111151200 2023.
Poornima Ramesh, Basile Confavreux, Pedro Goncalves, Tim P Vogels, and Jakob Macke. Indistinguishable network dynamics can emerge from unalike plasticity rules. bioRxiv, pp. 2023- 11, 2023.
Brian S Robinson, Theodore W Berger, and Dong Song. Identification of stable spike-timing dependent plasticity from spiking activity with generalized multilinear modeling, Neural com- putation, 28(11 1):23)0-2333 2016.
Louis K Scheffer, C Shan Xu, Michal Januszewski, Zhiyuan Lu, Shin-ya Takemura, Kenneth J Hayworth, Gary B Huang, Kazunori Shinomiya, Jeremy MaitiinSheepard, Stuart Berg, et al. A connectome and analysis of the adult drosophila central brain. Elife, 9:e57443, 2020.
Yichun Shuai, Areckul Hirokawa, Yulian Ai, Min Zhang, Wanhe Li, and Yi Zhong. Dissecting neural pathways for forgetting in Drosophila olfactory aversive memory. Proceedings of the National Academy of Sciences, 112(48):E6663- 72, December 2015.
Nicholas A Steinmetz, Christof Koch, Kenneth D Harris, and Matteo Carandini. Challenges and opportunities for large-scale electrophysiology with neuropixels probes. Current opinion neu- robiology, 50:92-100, 2018
12

---

Nicholas A Steinmetz, Cagatay Aydin, Anna Lebedeva, Michael Okun, Marius Pachitariu. Marius Bauza, Maxime Beau, Jai Bhagat, Claudia Böhm, Martijn Broux, et al. Neuropixels 2.0: A minia- turized high-density probe for stable, long-term brain recordings. Science, 37266339):eab45888 2021.
lan Stevenson and Konrad Koerding. Inferring spike-timing data. Advances in neural information processing systems, 24, 2011.
plasticity from spike train
Shin-Ye Ya Takemura, Kenneth J Hayworth, Gary B Huang, Michal Januszewski, Zhiyuan Lu, Eliz- abeth C Marin, Stephan Preibisch, C Shan Xu, John Bogovic, Andrew S Champion, et al. A connectome of the male drosophila ventral nerve cord. bioRxiv, pp. 2023-06, 2023.
Scbastian Thrun and Lorien Pratt. Learning to learn. Springer Science & Business Media, 2012.
Danil Tyulmankov, Guangyu Robert Yang, and LF Abbott. Meta-learning synaptic plasticity and memory addressing for continual familiarity detection. Neuron, 110(3):544-537, 2022.
Gilles C Vanwalleghem, Misha B Ahrens, and Ethan K Scott. Integrative whole-brain neuroscience in larval zebrafish. Current opinion in neurobiology, 50:136-145, 2018.
Ganchao Wei and lan H Stevenson. Tracking fast and slow changes in synaptic weights from si- multaneously observed pre-and postsynaptic spiking. Neural computation, 33(10):2682-2709, 2021.
Michael Winding, Benjamin D Pedigo, Christopher L Barnes, Heather G Patsolic, Youngser Park, Tom Kazimiers, Akira Fushiki, Ingrid V Andrade, Avinash Khandelwal, Javier Valdes-Aleman, et al. The connectome of an insect brain. Science, 379(6636):eadd9330, 2023.
Friedemann Zenke and Emre O Neftci. Brain-inspired learning on neuromorphic substrates. Pro- ceedings of the IEEE, 109(5):33--950, 2021.
Yan Zhang, Márton Rózsa, Yajie Liang, Daniel Bushey, Ziqiang Wei, Jihong Zheng, Daniel Reep, Gerard Joey Broussard, Arthur Tsang, Getahun Tsegaye, et al. Fast and sensitive gcamp calcium indicators for imaging neural populations. Nature, (7954)4))8884 891, 2023.
13

---

A Appendix
A.1 Glossary
Backpropagation through time: An algorithm for training recurrent neural networks by unrolling the network over time steps and applying the standard backpropagation algorithm, allowing the network to learn from sequences of inputs and capture temporal dependencies.
Binary Cross- Cross-Entrope Entropy (BCE): A loss function commonly used in binary classification problems that measures the difference between predicted probabilities and actual binary labels, encouraging the model to output confident and accurate predictions.
Connectome: comprehensive map of neural connections in the brain, detailing the structural con- nectivity between neurons or brain regions, providing insights into brain organization and function.
Covariance-based learning rule: A learning rule that updates weights based the covariance between neural activity and reward, adjusting connection strengths to capture statistical relationships in the data and potentially improving the network's ' performance.
Deviance explained: A statistical measure of goodness of fit for generalized linear models, quanti- fying the proportion of variability in the data that is accounted for by the model compared to a null model.
Gaussian process: A statistical model used for regression and probabilistic classification, defining distribution over functions and allowing for flexible, non-parametrie modeling with uncertainty quantification.
Generative adversarial network (GAN) approach: A machine learning framework where two neural networks compete to generate realistic data, with one network (the generator) creating syn- thetic samples and another (the discriminator) trying to distinguish between real and fake data.
Homeostatic adjustments: Changes in neural systems that help maintain stability, including meeh- anisms like synaptic scaling and intrinsic plasticity that regulate neuronal activity and prevent cessive excitation or inhibition.
Logistic regression: A statistical method for predicting a binary outcome by modeling the proba- bility of an event occurring as a function of input variables, using the logistic function to transform linear combinations of features into probabilities.
Synaptic plasticity:
The ability of synapses (connections between neurons) change their strength over time, forming the basis for learning and memory in the brain.
Truncated Taylor series: A mathematical method for approximating functions using polynomial terms, where the series is cut off after a finite number terms to balance accuracy and computational tractability.
subfield of machine learning focused on improving the learning process itself, developing algorithms that can learn how to learn and adapt quickly to new tasks or environments.
Meta-learning:
Higher-order plasticity where prior synaptic activity influences subsequent plastic- ity, regulating the threshold and magnitude of future synaptic changes to maintain network stability and optimize learning.
Metaplasticity:
Multilayer perceptrons (MLPs): type of artificial neural network with multiple layers of nodes, including input, hidden, and output layers, capable of learning complex non-linear relationships in data through backpropagation.
Mushroom body: A region in the insect brain involved in learning and memory, particularly impor- tant for olfactory learning, sensory integration, and decision decision-making processes.
Neuromorphic: Referring to artificial systems that mimic biological neural systems, often imple- mented in hardware to achieve brain-like computation with high efficiency and parallelism.
Neuropixels: type of high-density neural recording probe that allows simultaneous recording from hundreds of neurons across multiple brain regions with high spatial and temporal resolution.
14

---

Oja's rule: A famous local learning rule for neural networks that performs principal component analysis, extracting the dominant features from input data by updating synaptic weights based on the correlation between pre- and post-synaptic activity.
A.2 Implementation
In experimental contexts, the observed behavioral neural trajectories o( can span extensive time scales, often consisting of thousands of data points. Computing gradients over such long trajectories computationally demanding- Our framework implemented JAX (Bradbury al., 2018) and is designed to accommodate plasticity modeling across millions of synapses, anticipating future integration with large-scale connectomics datasets. terms of time, fitting Taylor coefficients on network with approximately 10 synapses and 1000-time step trajectories takes 2 hours NVIDIA H100 GPU.
A.3 Additional detail: Inferring a plasticity rule from neural activity
To create a ground-truth system with a known plasticity rule, we use single-layer neural network with 100 input neurons and N = 1000 output neurons to generate synthetic data. The output layer employs sigmoid activation function, chosen for its differentiability and biologically plausible output range of [0, IJ. We generate 50 training trajectories for the network. The input data sampled from Gaussian distribution with zero mean and variance of 0.1, independent of time. A subset of neurons, determined by the sparsity factor, is selected for the readout. The simulation runs over 50 time steps, caleulating the neural activity of all output neurons at each step. To ensure numerical stability and prevent exploding gradients, gradient clipping is applied with a threshold of 0.2.
The coefficients easy of the Taylor series expansion representing the plasticity rule are learned, initialized independently and identically distributed (i.i.d.) from a normal distribution with mean of and a variance of 10 Both the ground-truth and model network weights are initialized using Kaiming initialization from a zero-mean Gaussian distribution. Although these weights are drawn from the same distribution, they are resampled, resulting in different initial values. No regularization is applied during training. The Adam optimizer is used to train the weights of the plasticity model, with default parameters.
A.4 Additional detail: Inferring plasticity rules from behavior
In the behavioral simulation experiments, the ground truth plasticity rule is denoted as Tyr. We use a network with 2-10- 2-10-1 architecture and a sigmoid non-linee non-linearity. The plasticity MLP has a size of 4-10-1. The default L1 regularization is set to le-2, the moving average window is 10, and the input firing mean is 0.75.
Given the discrete nature of the observed behavior and the continuous output of the model, we employ the percent deviance explained as a performance metric. The percent deviance explained measures the model's 's ability to account for the variability in observed binary choices compared to null model that assumes no plasticity (i.e., the weights remain their initial random initialization). It represents the reduction in deviance relative to this null model, expressed as percentage. Higher values indicate greater liglliie,s,,,s, signifying a superior fit to the observed data.
Percent Deviance Explained = 100 x Deviancemodal
(1I)
Deviancemall
Since there are odors, they are encoded in a stimulus vector of dimension Odor corresponds to the first dimension "firing"; for these experiments, we use a value of 0.75, which we call the input firing mean (not to allow model to differentiate between x and x2 the plasticity rule). There is Gaussian noise with zero mean and a variance of 0.05 added to account for biological variability the signal. In one trajectory, traditionally there are 240 trials, consisting 3 blocks with different reward contingencies for odors in each block. The reward ratios are:
15

---

Odor A
Odor B
Block
0.2
0.8
Block 2
0.9
0.1
Block 0.2
0.8
Updates are performed on a single trajectory, with no batching. our simulations, we use 18 trajectories for training (matching the size of our previous experimental data) and 7 for evaluation for each seed. For each seed, results are reported as the median. Unless stated otherwise, all reported results averaged over 3 seeds.
A.5 Additional experimental parameters
In the following subsections we explore the effect of three factors (regularization, moving average window, input firing mean) on model performance. Only the parameter being tested varied while the other parameters held fixed at the previously defined values.
A.5.1 L1 regularization
We experiment with various values of the 1.1 regularization penalty applied to the Taylor coefficients. This encourages sparse plasticity solutions and prevents the coefficients from exploding into NaNs due to the learning of positive values that exponentially increase the synaptic weight the number of time points grows. do not apply LI regularization to the MLP parameters. The results of these experiments supports original choice of L1 regularization level (Fig. 5).
L1 Regularization
R² Weights
0.5
0.001
0.005
0.01
0.05
Figure 5: Effect of L1 regularization on R² of weights for Taylor plasticity rule
A.5.2 Moving average window
The moving average window refers the window size used for calculating the expected reward. For example, a moving average window of 10 would take the average of the rewards received over the last 10 trials. Our exploration of window size suggests that smaller windows allow our model to more accurately predict weights in behavioral simulation experiments (Fig. 6). This is to be expected as shorter historical dependencies on past trials reduce the noisiness of the expected reward estimates. Importantly however, our model still robustly identified learning rules when using out default choice of 10 trials which was guided by experimental results from Rajagopalan et (2023) that suggested that choice given trial was mediated by 10 past trials.
16

---

Moving Average Window
1
R Weights
0.5
5
10
20
Taylor
MLP
Figure 6: Effect of moving average window (used for calculated expected reward) the perfor- mance of learned plasticity rule
A.5.3 Input firing mean
This is the firing mean encoding used for representing odor. For example, a firing of represents the odor as [1, 0j. with additional Gaussian white noise fixed at variance of 0.05, Our model (specificallyy the MLP implementation) is able to account for a range of input firing means (Fig. 7).
Input Firing Mean
R² Weights
0.25
0.5
0.5
0.75
1.0
1.25
Taylor
MLP
Figure 7: Effect of input firing mean (used for odor representation) on the performance of learned plasticity rule
A.6
Validation on held-out data
We a validation experiment to assess the extent to which the fitting procedure could generalize to unseen biological data. Due to the sequential nature of the dataset, and that we are fitting our model individual flies' behavioral trajectories (i.e., sequences of choices), we cannot perform classical k-fold cross-validation in which a random subset of timepoints or trials are held out. Instead, we train model using the first 2% of individual fly's trajectory, and then we test on the last (100- 3%% (Figure 8). To ensure there is no data leakage from the training set, we re-initialize the model's s synaptic weights the beginning of each test sequence, although the test performance similar if the initial weights carried over from the final timestep of the training sequence.
17

---

% Deviance Explained
-10"
-10
Tain
0.05
0.1
0.25
0.25
0.3
0.35
0.4
0.45
0.7
0.8
0.85
0.55
1.0
Trajectory Cutoff
Figure 8: Percent deviance explained on the training and test data, training 2% of the fly data x%.
trajectory and testing on the remaining 100
Interestingly, we found that some flies had test performance close to training performance, whereas others had poor test performance (Figure 8). The flies with good test performance (defined having a positive percent deviance explained on the test set) reaffirmed our conclusion that weight decay term enabled a quantitatively better description of fly behavior (Figure 9, top). The flies with poor test performance (negative test percent deviance explained) also had negative plasticity coefficients for the w term (0o01 in Equation 6). suggesting that the differences in test performance were not due to different estimates this coefficient (Figure 9, bottom). More analyses are required determine why some flies had markedly better test performance than others.
A.7 Additional plasticity rules
In these additional experiments, we maintain the reward term as the difference from the expected reward. This approach facilitates bidirectional plasticity. Additionally, we incorporate weight decay term, experimenting with several coefficients, ultimately choosing value of 0.05 as it seems reasonable for our experimental configuration. Table presents comparison between the MLP and the Taylor series, reporting the Rt over weights, R² over activity, and the percent deviance explained. At a high level, both methods appear perform similarly. To gain deeper insights into why certain rules are more "recoverable" than others, an examination of weight dynamics for each method is necessary. The Taylor series model has 81 trainable parameters, while the MLP has 61. Fitting the rules with relevant subset of the Taylor series, selected through biological priors as done in the Drosophila experimental data, is expected to result in better performance.
18

---

Weight decay coefficient for flies with good test performance
0.00
-0.25
0.50
-0.75
-1.25
-1.50
Weight decay coefficient for flies with test performance
0.00
-0.25
-050
-1.25
-1.50
0.05
011
0.15
0.35
0.65
0.75
0.8
0.85
0.95
Trajectory cutoff
Figure 9: Learned weight decay term in the Taylor series parametrized plasticity rule (Equation 6),
from flies with positive (top) and negative (bottom) test set percent deviance explained.
19

---

Table 3: Evaluation of various different reward-based plasticity rules: R scores for weight and individual neural activity trajectories, and percentage of deviance explained for behavior.
MLP
Taylor
Plasticity Rule
Weights
R Activity
% Deviance
R Weights
R Activity
% Deviance
W 0.05r
0.27
0.34
79.92
0.41
0.51
84.22
"yWIghT 0.05r
0.61
0.67
84.39
0.33
0.30
82.79
ZyWir"² 0.05r
0.66
0.66
84.75
0.42
0.47
79.33
"y 0.05r
0.40
0.57
80.56
0.40
0.48
79.03
Zyr² 0.05r
0.75
0.73
70.78
0.66
0.61
70.65
" 0.05
0.59
0.68
70.41
0.59
0.69
68.84
"/ - 0.05
0.59
0.62
64.98
0.55
0.62
64.44
Σ₁ 0.0 0.05%,yyr
0.81
0.96
63.64
0.81
0.95
64.01
2, 0. 0.05a, WiyT
0.84
0.95
64.19
0.82
0.95
63.24
E,r 0.05x,3K RS
0.80
0.96
62.90
0.76
0.94
62.50
0.85
0.96
64.76
0.78
0.94
61.91
ZGr 0.05r 2yr 0.05% 1 Wyy
0.70
0.76
62.01
0.63
0.68
61.82
0.82
0.96
63.13
0.73
0.93
61.05
2 O. 0.05Ir, Wyy
0.83
0.95
61.24
0.81
0.95
60.85
Frr 0.05 W.
0.84
0.96
63.53
0.72
0.92
60.33
2 4 0.05xr,
0.89
0.97
61.99
0.77
0.91
60.05
2 0. 0.05.ry Wiy
0.89
0.95
54.92
0.88
0.94
53.94
0.05.zy14 2yr²
0.79
0.95
56.40
0.77
0.92
53.76
x 2
0.89
0.96
54.32
0.81
0.92
52.87
22 0.05wa
0.87
0.91
57.01
0.70
0.86
51.80
0.05æ, yhr
0.84
0.96
53.14
0.78
0.93
51.61
2rr
0.05æ,yya,,
0.81
0.95
54.92
0.76
0.93
51.52
xyr²
0.05
0.85
0.96
53.04
0.78
0.92
51.30
2,r² 0. 0.05a WDWW
0.90
0.96
53.90
0.82
0.92
51.29
2 0.05x,
0.89
0.95
51.52
0.85
0.93
51.03
2 0.05xzrr
0.88
0.96
54.17
0.79
0.93
50.90
g5 0. 0.05
0.91
0.95
52.94
0.84
0.91
50.24
4 0.05y:r
0.94
0.95
52.84
0.86
0.92
50.20
25r 0.05wiyy
0.87
0.94
49.72
0.82
0.92
48.90
Z5r w5
0.94
0.96
48.58
0.90
0.95
48.60
0; L 2 0.05
0.66
0.90
47.26
0.55
0.84
47.00
T, 0.05wa
0.86
0.94
46.44
0.78
0.91
46.26
2,r 2 w.05
0.91
0.92
43.64
0.89
0.93
44.04
4,r" 0.05g,r
0.96
0.97
43.74
0.93
0.95
43.18
x,r² 0.05waj
0.92
0.93
42.68
0.91
0.94
42.43
2 y W, T2 0.05r
0.65
0.53
43.26
0.65
0.57
42.32
2 0.05y:
0.93
0.92
44.13
0.91
0.94
41.69
y: W. r 0.05
0.69
0.89
38.52
0.61
0.85
37.74
3 0.05
0.52
0.87
37.01
0.60
0.82
36.81
22r² 0.05y:
0.97
0.97
34.55
0.96
0.97
34.36
ur 0.05r
0.72
0.65
33.94
0.68
0.60
33.57
25 W:WW 0.05r
0.60
0.53
34.82
0.65
0.57
33.45
W.W.jT 0.05x,rr
0.04
0.45
28.56
0.08
0.46
28.63
<Jy. 0.05r
0.50
0.44
28.81
0.70
0.60
28.08
y:Wrrr2 0.05Irr
0.17
0.47
26.90
0.31
0.44
24.92
y:Wrr 0.05r
0.38
0.32
16.49
0.60
0.59
15.87
y 0.05r
0.01
0.01
14.06
0.24
0.29
13.62
yzy? WigT - 0.05æ,r
0.82
0.94
4.64
0.86
0.95
4.35
2/ win 0.05%yr
0.78
0.94
4.31
0.90
0.96
4.01
TST- - 0.05I,r
0.75
0.92
4.30
0.92
0.96
3.96
2,0, Wig 1 0.05x,r
0.79
0.93
4.39
0.89
0.97
3.87
K,W:WiTT 0.05zyr
0.70
0.91
3.49
0.87
0.96
3.29
25 0.05x,rr
0.63
0.90
3.28
0.83
0.96
3.04
20

---

B NeurIPS Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Ye]]
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: Yes]
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [No]
Justification: All experiments are reproducible, and code is opensource. We report the rough time it takes run experiments, but have not provided extensive indepth details like the time of execution, etc.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics attps://neurip:: /ttph//th//////ppoop...c///////p
Answer: [Yes]
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [NA]
21

---

Justification: We have discussed the impact that our paper could have within the field of neuroscience. Larger societal impact of this work however is unclear, and likely will only be felt down the road as this approach is used more widely, we have therefore not dedicated a section discuss this.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text instructions given to participants and screenshots, applicable, well as details about compensation (if any)?
Answer: [NA]
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
22


Reviews:
[{"review_id": "slzk0Gc5p2", "reviewer": "Reviewer_2Jtn", "summary": "This paper presents a reward learning-based method for recovering biological synaptic plasticity rules in a model network. It is meant to be applied to both neural and behavioral data. The method consists of taking real learning trajectories collected in response to stimuli (of some observable metric modality) and showing those stimuli to the model artificial neural network, then computing the loss between the generated and. modeled trajectories and taking a gradient step in the direction of the ground truth data. The weight update is determined by some plasticity function, represented by either a Taylor expansion or an MLP, and also learned over the course of training according to the trajectory loss. After training the model network, various plasticity rules are tested by analyzing the learned parameter values of the plasticity function. \n\nAfter establishing the method, the paper presents specific experimental settings and results. It begins with a simpler case of simulated neural activation data with plasticity dynamics based on Oja's rule, MSE loss between the two trajectories, and a Taylor series plasticity function. The paper shows that Oja's rule can be recovered - it is seen clearly in the learned parameters of the Taylor series plasticity function. Ablations are also conducted over increasing sparsity and noise, showing degradation. \n\nThe paper then goes on to a simulated behavioral data setting with a small MLP representing the plasticity function. Here, it establishes percent deviance explained between the ground truth and modeled trajectories as a performance metric. Finally, it demonstrates some proof of concept in actual drosophila behavioral data, including a forgetting mechanism (negative dependency on the weights, not just the reward signal and presynaptic activations). \n\nFinally, it goes through related work.", "questions": "- How should I ground my judgement of the percent deviance explained results?", "limitations": "Limitations - yes\nSocietal impact - they include the NeurIPS checklist in their supplementary and say the societal impacts have been addressed, but they haven't anywhere, nor is there an assertion that they don't need to. They probably don't, though.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "### Originality\nAs far as I am aware, this particular reward-based training set up with an organism model and a separately trained plasticity model is novel.\n\n### Significance\nIf viable, this would be a very useful work for modeling. This is an initial exploration, but the significance of the results is nontrivial - it opens up a research direction.\n\n### Quality\n- Training setup is creative! It would be nice to know how well this scales, as trajectory-based sequence learning is difficult. \n- Experimental settings are thoughtful and elucidating\n  - The toy example does an especially good job of helping the reader build intuition\n  - Showing that synaptic trajectory error improves over time even though training is done with neuronal trajectories - this is clever and convincing\n  - Real drosophila experiment is exciting \n- Analysis of plasticity functions, how they show up in parametrization, and what that means, is very itneresting\n\n### Clarity\n- Results has a claims-driven structure that is helpful for understanding takeaways\n- Methods are very clear", "weaknesses": "### Clarity\n- Paper is very jargon-heavy without defining niche terms and having clear conclusions in paragraphs. It would help to have simpler language and/or more explanation\n- Percent deviance explained - we don't get grounding or baselines to understand how to judge these results. Are there baseline methods, or any kind of comparison point/context?\n\n### Quality\n- It's not clear how far this method, or even the principles established in the paper, can take us. \n  - The space of tested plasticity rules seems hand-designed and limited. Rules are rejected only when they can be clearly defined and tested. \n  - Percent deviance explained results seem to leave plenty of room for improvement \n  - Test setups are quite simple. This is mitigated highly by the presence of real drosophila behavior data - that experiment is really useful - but still, simple and limited, particularly in the underlying dynamics \n  - The paper argues that it is \"reasonable\" to assume that we can have the modeling network architecture exactly match the true architecture, because of available connectome data. Connectomes are static and extremely detailed, and there are lots of elements (e.g. immense recurrence) that we do not have architectures for. This is again very much improved by the real drosophila data experiment, but it is nevertheless concerning. \n  - Underlying plasticity rules in synthetic settings are very simple, even in the case being learned by an MLP. The drosophila behavioral experiment helps because we see the discovery of the forgetting rule, but we don't have any sense of a full space to look for, or even a partial but extensive space to look for (things that both should and shouldn't show up). Even the \"forgetting\" rule discovery is believable but requires a lot of interpretation of results. \n- Ablations aren't really discussed. Noise and sparsity make sense as ablation factors, but what do the results mean beyond just \"noise and sparsity cause problems\"? Especially because their effects are strangely similar.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer 2Jtn\n\nThank you for your constructive feedback on our manuscript. We are glad that you found our work to be “creative,”  and “clever and convincing.” Below, we address each of the concerns you expressed in your review, and hope that you will find our revised manuscript to be improved.\n\n## Clarity\n\n1. We have added conclusions to paragraphs where this was missing and simplified language that was  jargon-filled. These changes are highlighted in red. To help readers understand any remaining specialized language, we added a Glossary before the Appendix. If the Reviewer alerts us to additional terms that they found to be jargon, we would be happy to either eliminate or add it to our Glossary depending on its utility.\n\n2. (Also the response for Question 1) One should interpret the percent deviance explained similarly to the R-squared metric. A value of 0% corresponds to chance performance and a value of 100% corresponds to theoretically optimal performance. It’s useful to our paper because it doesn’t assume continuous outputs or Gaussian noise. It thus applies to binary behavior more naturally than the R-squared metric does. A definition of this metric is provided in Appendix section A3 and we have added a reference to this in the text. We apologize that this reference previously pointed to an incorrect section of the paper. \n\n## Quality\n\n1. We agree that the space of plasticity rules is hard to specify and sample in its entirety. We’d like to clarify two related issues.\n    - First, we acknowledge that when choosing plasticity rules to simulate ground-truth data, we emphasized canonical models that have been “hand-designed” by the computational neuroscience community due to their  properties and biological relevance. Their simplicity reflects the current state of understanding in the field. However, we also simulated a large number of other rules in the Appendix (Table 2) excluding the unstable rules. We chose this naive sampling of plasticity rules because the field does not yet have the knowledge to know what class of rules are most relevant. Given this, a better approach was not clear to us as that the space of simulated rules cannot be sampled exhaustively. \n\n    - Second, we’d like to note  that our framework and fitting procedures were designed with the goal of leveraging large-scale biological data to help move beyond hand-designed rules. Our paper’s main assumption is that the plasticity rule is a function of only presynaptic activity, postsynaptic activity, reward, and the current weight value. Our fitting procedures are designed to estimate this unknown function from data. There are multiple ways to parameterize functions, and we consider two parameterizations with complementary strengths. First, we use a low-order polynomial, which can be taken as estimating the Taylor series for the function and is easy to interpret as it relates simply to the canonical rules. Second, we use a multilayer perceptron, which sacrifices interpretability for expressivity. In either case, we’re providing a general parameterization that can reveal unexpected results when fit to biological data. \n\n2. The Reviewer is correct that our method sometimes infers plasticity rules that poorly approximate the weight, activity, and/or behavioral trajectories (Table 2). This isn’t surprising because plasticity rules are nonlinear dynamical systems, and their predictions may show sensitivity to parameters and initial conditions. We would contend that no inference method will be able to perfectly recover an arbitrary plasticity rule. The intention of Table 2 is to merely illustrate the range of possibilities. Nevertheless, we hypothesize that the rules implemented by biology are less unwieldy than some possibilities in Table 2. Indeed, our method works very reliably on the canonical rules that were designed to capture biological phenomena. We hope that using our framework to fit real data will help us to identify novel, well-behaved rules without necessitating laborious hand design.\n\n\n3. As explained above, we acknowledge that we’ve emphasized tests against simple, plausible learning rules. However, the suite of learning rules in Table 2 include a wide range of ground-truth plasticity rules with nonlinear (polynomial) dependencies among the terms. We agree that applying our method to real data is a productive path forward, and found it very encouraging that the method already uncovered something novel and interpretable in the Drosophila data. We anticipate more discoveries with richer datasets and models.  \n\n4. The reviewer’s comment regarding connectomics is addressed in the general response to reviewers.\n\n5. We believe this comment on the simplicity of the synthetic rules has been adequately addressed by our earlier responses.\n \n6. We have added text to Section 3.1 to clarify the interpretation of our choice of ablation experiments. This was guided by the kinds of incomplete information we would likely deal with when applying our model to biological data. Often neural recordings are noisy and even the most state-of-the-art recording tools  suffer from not being able to record entire populations of interest. Our ablation experiments quantify how resilient our approach is to these sources of error that are often present in real data. \n\n## Limitations\n\n1. We have changed our response to the societal impact question of the NeurIPs checklist and provide a justification for why this is not discussed in the main text. While we have discussed our paper’s impact within the field of neuroscience, the larger societal impact will likely only be revealed as the method is used more widely to understand specific neural systems and their plasticity rules. As an immediate societal impact is exceedingly unlikely, we did not dedicate a section to discuss these issues.\n\n2. A description about the scalability of our approach has been provided in the general rebuttal to all reviewers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m7ukxcdvBU", "reviewer": "Reviewer_5Cze", "summary": "This paper studies how to learn local learning rules (like those plasticity rules thought to be used by real neurons) in a data-driven way from neural activity or behavioral timeseries. This is applied to simulated as well as fly behavioral data. The true learning rules can be learned from synthetic activity timeseries when complete observations are available (Fig 2), whereas from behavior alone some more error is incurred (Fig 3). When using fly behavioral data, the authors get better fits than previous methods (Fig 4), and a claim is made that the \"forgetting term\" in the resulting fit is important.", "questions": "* What justification do you have for claiming that your model requires less energy than an LLM of the same size? (Line 45-46)\n* Can you be precise about which term in Fig 4C is the decay term? I think you mean 000. It isn't evident whether that is significantly different from 0. The error bars are quite wide.\n* Have the authors considered the vast literature on fitting dynamics themselves from timeseries? I am familiar with the SINDy method, which uses polynomial or other families of basis functions and includes a LASSO-type penalty to fit dynamics. It is likely that some regularization or penalty could improve the learned models in your setting, too. (I now see this buried in the appendix, but it should be discussed in the main text.)\n* Please discuss why you did or didn't use cross-validation", "limitations": "I think it's a stretch to claim that the underlying connectivity (presence of nonzeros in weight matrix) is known because of connectomics (lines 80-82). That may be true for some small model organisms like Drosophila, and only just, but it is nowhere near true for others such as the mouse. The authors should be careful about making claims of identifiability with models such as theirs. For instance, it's known that incomplete knowledge of the connectome or partial observations of dynamics can lead to spurious correlations. An example that comes to mind is https://journals.aps.org/pre/abstract/10.1103/PhysRevE.109.044404 .\n\nYour experiments actually show this non-identifiability already: In figure 2G we see that the Oja rule is not recovered when the observations are incomplete. There is a brief discussion of this \"degeneracy\" in the conclusions and limitations section, but I think it is a bigger issue than the authors are claiming. In particular, this conjecture that \"in the infinite data limit the fitted plasticity rules would, in fact, be unique\" could easily be wrong. It is certainly not a useful argument to the experimentalist who will always have limited data, especially if it's behavioral or brain recording data.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper studies an interesting inference problem which is relevant to both AI and neuroscience. It is well-written with a discussion of some limitations. It studies both synthetic and real data. I found the idea of trying to fit this kind of model to behavior interesting. I wouldn't have thought that it would be possible, to be honest.", "weaknesses": "I think the main weakness is what I identified below in limitations: the degeneracy or non-identifiability of such models. I can imagine that even in settings where the equations are the correct model that it is not identifiable without some regularization.\n\nThe model fits to behavioral data are given as R^2 values on the behavior themselves. It does not seem that any cross-validation was used for model fitting, so there is a possibility that your model is overfitting. This should be discussed.\n\nI'll admit I'm not super familiar with existing work in this area, so I am not sure how novel these results are relative to what's been tried before.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer 5Cze\nDear Reviewer,\n\nThank you for your careful reading of our work and insightful comments. We are glad that you found the paper to be “interesting,” and “relevant.” We address each of the comments mentioned in your Weaknesses, Questions, and Limitations sections below. \n\n## Weaknesses\n1. We have tried to distinguish between a model being non-identifiable, by which we mean that its parameters are impossible to uniquely infer, and sloppy, by which we mean that its parameters are poorly constrained by available data. \n\n2. Regarding sloppiness, we agree that it could be hard to estimate the parameters of the plasticity rule in practice given finite data sets. Multiple terms in the plasticity rule may lead to similar weight changes. Nevertheless, our paper shows that it is empirically possible to infer plasticity rules given reasonable amounts of simulated data. It also identifies a biologically interpretable but previously unknown term in the plasticity rule of the fly mushroom body. This term would not appear consistently in the statistical fits of individual flies if it was too sloppy to be determined. Simply, our method is already useful, regardless of whether it can detect the sloppiest components of plasticity rules. \n\n3. Regarding identifiability, if one could measure every term that enters the plasticity rule, then the identifiability of the plasticity rule would follow from the uniqueness of Taylor series expansions. Since weights are assumed to be unmeasured, then it’s in principle possible that some plasticity rules may be non-identifiable. This seems unlikely to us, because each component of the plasticity rule contributes to weight changes that measurably affect the postsynaptic activity. Nevertheless, we acknowledge that we don’t have a formal mathematical proof that the plasticity rule is identifiable with unmeasured synaptic weights. \n\n4. In our opinion, identifiability is less relevant than sloppiness, and sloppiness is less relevant than our empirical demonstrations. Our method already works well enough to be applied fruitfully to simulated and real-world problems. \n\n5. We acknowledge the reviewer's concern regarding the potential for overfitting due to the absence of cross-validation. We want to clarify that in our simulated experiments we consistently utilized a separate held-out test set to evaluate model performance - we have clarified this in the main text. However, we recognize the value of incorporating cross-validation for fitting the fly experimental data. If the paper is accepted for publication, we will implement a cross-validation procedure and include it in the final version of our manuscript. We agree with the reviewer that this would substantially enhance the robustness of our model evaluation.\n\n## Questions\n1. We appreciate the reviewer's observation and have removed the statement about energy implications for training LLMs, as upon further consideration the point seemed tangential to the goals of our paper. \n\n2. We have added text to the caption of Fig 4C to clarify that parameter 001 corresponds to the weight decay term, and have listed what each of the terms mean. The 000 term is a bias.\n\n3. We thank the reviewer for pointing us to the literature that focuses on fitting dynamical models from time-series data, especially the idea that regularization penalties can substantially improve model performance. As the reviewer pointed out, we have begun exploring the L1 regularization penalty in our model, and we are familiar with the related SINDy approach. If accepted, we will incorporate a more thorough investigation of the impact of regularization on model performance, and we will move the relevant text discussing this from the Appendix to the main manuscript.\n\n4. We have already addressed this concern in the “Weaknesses” section of our response.\n\n\n## Limitations\n\n1. The reviewer’s comment regarding connectomics is addressed in the general response to all reviewers.\n\n2. We have already discussed identifiability and sloppiness in our response to the Reviewer’s “Weaknesses” comments. Here the Reviewer additionally brings up “model mismatch,” by which we mean that best-fit model parameters are not directly interpretable if the model parameters are out of accordance with the underlying biology. We agree that this is very often an issue with model fitting, and we certainly don’t claim to have a solution to this general problem. We don’t currently see anything that we could add to the study that would alleviate this concern, and our approach has instead been to illustrate related failure modes. Indeed, the Reviewer points to our figures to illustrate the point, which we think indicates that we’ve given it fair treatment. We are happy to make additional revisions if the Reviewer could clarify what they find problematic about our treatment. The quoted text that the Reviewer provides is about “identifiability,” not “model mismatch,” and we think it is important to keep the related issues of identifiability, sloppiness, and model mismatch distinct as these issues are fundamentally different.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oIdEensQVa", "reviewer": "Reviewer_MyoH", "summary": "This paper proposes a novel method for inferring plasticity rules from neural and behavioral data. In contrast to previous approaches, the plasticity rule is directly optimized to maximize the similarity of the output of a model trained with the plasticity rule to a target (neural activity or behavior). This approach is validated with synthetic experiments and used to infer plasticity rules from fruit fly behavioral data.", "questions": "* Double-check the subscripts in Eq (5), I think $y_j$ should be $y_i$.\n* For the bottom portion of Fig 2, are the weights being compared the weights after training when using the ground truth and inferred rules?\n* Is there some explanation for why the last rule in Table 1 exhibits such different behavior?\n* Is there held out validation data in the experiments in Section 5, or is a single dataset used throughout?", "limitations": "Yes", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* Determining the functional roles of synaptic plasticity is a fundamental problem in computational & theoretical neuroscience.\n* The method is not particularly novel, but it is simple, nicely explained, flexible, and well-validated.\n* The paper is very well-written, and all methods and findings are clearly described.", "weaknesses": "* My sole concern with this method is its broader applicability to more complex problems; inferring the learning dynamics involved in a simple forced choice task where the biological network is relatively well-known is very different from partially observed neural activity with poorly-understood connectivity during free behavior. I'm particularly concerned about the case where the influence of a large number of unobserved neurons needs to be considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer MyoH\n\nDear Reviewer,\n\nThank you for your feedback and your recognition of the interesting aspects of our paper. We have addressed your highlighted Weaknesses and Questions one by one below:\n\n## Weaknesses\n1. We acknowledge that when choosing problems to test our approach we used behavioral settings and plasticity rules that were relatively simple. Our choice of problems was inspired by the kinds of behavioral data that our colleagues had available to test our approach on, and it is something that can be expanded upon in the future to include more complex problems. Despite this, we believe that the experiments described in this paper suggest that our approach can in fact be applied to more complex situations. Our reasoning for this is two-fold:\n    - First, while we made the theoretical assumption  in this paper that the plasticity rule is a function of only presynaptic activity, postsynaptic activity, reward, and the current weight value, our fitting procedures are designed such that additional terms can be added without difficulty. This would allow us to estimate a large variety of plasticity rules. \n    - Second, we simulated and were able to retrieve a large number of basically arbitrary plasticity rules (in the Appendix) from neural trajectories. This estimation also showed resilience to noise and sparsity of the input data. \n\n## Questions\n1. You are correct that  the subscripts in Eq (5) had a typographical error. We have updated the text to reflect this. Thank you for pointing this out.\n\n2. Regarding the R-squared calculation in Figures 2E and 2F, we calculated the R-squared score by comparing the weight trajectories of the ground truth rule with the weight trajectories of the inferred rule after the learning rule optimization was completed. In particular, our R-squared comparison was not limited to the final weights, and it instead compared the full trajectories.\n\n3. The Reviewer is correct that our method sometimes infers plasticity rules that poorly approximate the weight trajectories, activity trajectories, and/or behavioral trajectories (Tables 1 and 2). This isn’t surprising because plasticity rules are nonlinear dynamical systems, and their predictions may depend very sensitively on parameters and initial conditions. We would contend that no inference method will ever be able to perfectly recover an arbitrary plasticity rule, and the intention of Tables 1 and 2 is to merely illustrate the range of things that are possible. Nevertheless, we would hypothesize that the rules implemented by biology are less unwieldy than some of the possibilities revealed by the tables. Indeed, our method works very reliably on the canonical rules that were designed to do useful computations and/or capture biological phenomena. We hope that using our framework to fit real data will help us to identify novel, well-behaved rules without necessitating laborious hand design.\n\n4. In Section 5, we fit behavioral data from 18 flies, utilizing the full dataset for our reported results. We acknowledge, however, that incorporating cross-validation would strengthen our analysis. If our paper is accepted for publication, then we commit to implementing this approach and including it in the final version of our manuscript, thereby enhancing the robustness of our model evaluation. It's worth noting that in our simulation experiments, we employ a separate held-out test set to assess model performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# General response to reviewers

We appreciate the thoughtful and constructive feedback provided by all reviewers. We have revised our manuscript based on this. This rebuttal addresses two key points relevant to all reviewers.

## **1. Scalability of the method**

In response to Reviewer 2Jtn's interest in knowing "how well the method scales as trajectory based learning sequence learning is difficult", we share the following hot-off-the-press *preliminary* results. Our primary results (Figures 3, 4, Table 1, and supplementary materials) used a trajectory length of 240 and a [2-10-1] neural network architecture. To address scalability, we conducted additional analyses varying both trajectory length and hidden layer size. These analyses used simulated data with a ground truth rule of x.(r - E(r)) on behavioral data, employing a Taylor series plasticity function parameterization. Results were averaged over three seeds for robustness.

It's noteworthy that for a [2-1000-1] architecture, our method performs backpropagation through time over 2000 synapses across 240 time points, with each synapse updated at every time point following the parameterized Taylor expansion. This demonstrates that computational complexity increases with both network size and trajectory length.

### **A) Scalability with trajectory length**

| Trajectory length | 30   | 60   | 120  | 240  | 480  | 960  | 1920 |
|-------------------|------|------|------|------|------|------|------|
| R2 Activity       | 0.92 | 0.91 | 0.91 | 0.94 | 0.95 | 0.87 | 0.92 |
| R2 Weights        | 0.79 | 0.74 | 0.70 | 0.78 | 0.72 | 0.53 | 0.64 |
| Percent Deviance  | 39.52| 40.14| 48.06| 61.91| 76.90| 73.78| 79.66|

The model's goodness-of-fit generally improved with longer simulations, likely due to more data points for inferring the plasticity rule. However, R-squared values for activity and weights peaked before declining, suggesting potential overfitting on very long trajectories. We plan to add cross-validation analysis in the final manuscript if accepted. The current trajectory length (240) appears near optimal for R-squared values, mitigating overfitting concerns for the main results.

### **B) Scalability with network architecture**

Our primary findings use a [2-10-1] architecture (20 synapses updated at every time point). We've demonstrated that the framework scales to 1000 hidden units (2000 synapses).

| Hidden Layer Size | 10    | 50    | 100   | 500   | 1000  |
|-------------------|-------|-------|-------|-------|-------|
| R2 Activity       | 0.94  | 0.94  | 0.95  | 0.95  | 0.95  |
| R2 Weights        | 0.78  | 0.75  | 0.79  | 0.79  | 0.79  |
| Percent Deviance  | 61.91 | 62.29 | 62.25 | 62.27 | 62.26 |

Model performance remains consistent when scaling to larger synapse counts, assuming the same plasticity rule is applied.

We are currently conducting additional experiments on scalability with respect to the parameters in the plasticity function, focusing on:
1. Plasticity MLP complexity (size)
2. Number of terms in the Taylor expansion (up to 3^4)

These findings will be included in the camera-ready version of the manuscript, if accepted.

## **2. Connectomics and model architecture**

We acknowledge the reviewers' correct observation that connectomes miss important architectural information and are most complete only in small model organisms. We will revise the text to avoid overstating the relevance of this data for larger brains.

Despite these limitations, several research groups have successfully used connectomics to build biologically realistic network models by parameterizing and fitting the most important unknown quantities [1-3]. For example, while connectomes provide synapse counts but not weights, accurate neural network models have been built by fitting cell-type-specific scale factors that convert synapse counts to weights.

Dynamical neural network models could be similarly designed to include synaptic plasticity:
1. The connectome could be used to infer an adjacency matrix at the level of cell types or single cells.
2. Plasticity rules could be parameterized, as in this paper, and fit to neural/behavioral dynamics.
3. The probability distribution of synapse counts could be used to constrain the probability distribution of synaptic weights, providing constraints on the plasticity rule.

We have two main reasons for assuming it's reasonable to match the architecture to connectomics data:

1. In Section 3, our intention was to demonstrate that our approach can solve the inference problem in a setting where the predictive and generative network architectures matched. This scenario is indeed aspired to by fly neuroscientists in the era of connectomics. As the reviewers point out, we move away from this assumption when modeling the behavioral data in Sections 4 and 5.

2. We argue that the available connectomic information is sufficiently rich to allow us to construct neural networks similar to the structure of brain regions of interest. In Section 5, we show that such a mushroom-body-inspired architecture allows us to infer rules that agree with predominant ideas in the field and contribute unique and interpretable additions to the knowledge base.

We have added language to clarify that while we believe using connectomic information to design model architecture is reasonable, mismatches in generative and model architectures can lead to errors in interpretation. These should be taken into consideration, and we have added a citation to the reference suggested by reviewer 5CZe.

**References**

[1] Lappalainen et al. *Connectome-constrained deep mechanistic networks predict neural responses across the fly visual system at single-neuron resolution*, (bioRxiv, 2023)

[2] Mi et al. *Connectome-constrained Latent Variable Model of Whole-Brain Neural Activity*, (ICLR, 2021)

[3] Beiran et al. *Prediction of neural activity in connectome-constrained recurrent networks*, (bioRxiv, 2024)

Meta-review: The paper proposes a nice approach for inferring synaptic plasticity rules from (partially) observed neural activity or behavioral read-outs. The approach is fairly straightforward, but it seems to work well across several examples. I found it surprising and impressive that behavior in a binary decision making task could say much about the nature of the plasticity rule. 

The reviewers raised some very good points, and I encourage the authors to incorporate their feedback and make the promised changes.
Decision: Accept (poster)