Model Based Inference of Synaptic Plasticity Rules

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 00:40:17

Abstract: Inferring the synaptic plasticity rules that govern learning in the brain is a key challenge in neuroscience. We present a novel computational method to infer these rules from experimental data, applicable to both neural and behavioral data. Our approach approximates plasticity rules using a parameterized function, employing either truncated Taylor series for theoretical interpretability or multilayer perceptrons. These plasticity parameters are optimized via gradient descent over entire trajectories to align closely with observed neural activity or behavioral learning dynamics. This method can uncover complex rules that induce long nonlinear time dependencies, particularly involving factors like postsynaptic activity and current synaptic weights. We validate our approach through simulations, successfully recovering established rules such as Oja's, as well as more intricate plasticity rules with reward-modulated terms. We assess the robustness of our technique to noise and apply it to behavioral data from \textit{Drosophila} in a probabilistic reward-learning experiment. Notably, our findings reveal an active forgetting component in reward learning in flies, improving predictive accuracy over previous models. This modeling framework offers a promising new avenue for elucidating the computational principles of synaptic plasticity and learning in the brain.

Main paper: 
 introduction: 
 While the backpropagation algorithm has been a cornerstone in the training of artificial neural networks (Rumelhart et al., 1986) , its biological implausibility limits its relevance for biology (Lillicrap et al., 2020) . Specifically, backpropagation involves non-local weight updates and symmetric feedback connections, mechanisms that are not observed in biological neural systems. This leads to a compelling question: how do biological brains, devoid of backpropagation, accomplish learning? Synaptic plasticity, the ability of synapses to change their strength, is a key neural mechanism underlying learning and memory in the brain. These synaptic updates are driven by local neuronal activity and global reward signals, and they in turn modify the dynamics of neural circuits. Advances in neuroscience have made it possible to record neuronal activity at an unprecedented scale (Steinmetz et al., 2018; Vanwalleghem et al., 2018; Zhang et al., 2023) , and connectome data for various organisms is becoming increasingly available (Bentley et al., 2016; Hildebrand et al., 2017; Scheffer et al., 2020) . However, the inaccessibility of direct large-scale recordings of synaptic dynamics leaves the identification of biological learning rules an open challenge. Existing neuroscience literature (Abbott & Nelson, 2000; Morrison et al., 2008) suggests that synaptic changes (∆w) are functions of local variables such as presynaptic activity (x), postsynaptic activity (y), and current synaptic weight (w), as well as a global reward signal (r). Uncovering the specific form of this function in different brain circuits promises deep biological understanding and holds practical significance for inspiring more biologically plausible learning algorithms for AI, in particular with neuromorphic implementations (Zenke & Neftci, 2021) .In this paper, we bridge computational neuroscience and deep learning to introduce a general framework for inferring synaptic plasticity rules. Our method optimizes plasticity rules to fit both neural and behavioral data, thereby shedding light on the mechanisms governing synaptic changes in bio-logical systems. Our approach employs interpretable models of plasticity, parameterized to enable a direct comparison with existing biological theories.In summary, our key contributions are:• Introduction of a versatile framework for inferring synaptic plasticity rules from experimental data, accompanied by an open-source, extensible implementation.1 • Demonstration of a gradient descent based approach for recovering plasticity rule parameters solely from observed neural activity or probabilistic behavior, along with a characterization of rule recoverability.• Application of our model to behavioral data from fruit flies, revealing an active forgetting mechanism in mushroom body circuits for decision making, learning, and memory. 
related work: 
 Recent work has begun to address the question of understanding computational principles governing synaptic plasticity by developing data-driven frameworks to infer underlying plasticity rules from neural recordings. Lim et al. (2015) infer plasticity rules, as well as the neuronal transfer function, from firing rate distributions before and after repeated viewing of stimuli in a familiarity task. The authors make assumptions about the distribution of firing rates, as well as a first-order functional form of the learning rule. Chen et al. (2023) elaborate on this approach, fitting a plasticity rule by either a Gaussian process or Taylor expansion, either directly to the synaptic weight updates or indirectly through neural activity over the course of learning. Both approaches consider only the difference in synaptic weights before and after learning. In contrast, our approach fits neural firing rate trajectories over the course of learning and can be adapted to fit any parameterized plasticity rule.Other work infers learning rules based on behavior instead. Ashwood et al. (2020) uses a Bayesian framework to fit parameters of learning rules in a rodent decision-making task. The authors explicitly optimize the weight trajectory in addition to parameters of the learning rules, requiring an approximation to the posterior of the weights. Our approach directly optimizes the match between the model and either neural or behavioral data, as defined by a pre-determined loss function. Interestingly, despite this indirect optimization, we see matching in the weight trajectories as well. Rajagopalan et al. (2023) fit plasticity rules in the same fly decision-making task we consider here. They assumed that the learning rule depended only on presynaptic activity and reward, which recasts the problem as logistic regression and permits easy optimization. Our approach allows us to account for arbitrary dependencies, such as on postsynaptic activities and synaptic weight values, and we thereby identify a weight decay term that leads to active forgetting.Previous work has also considered inferring plasticity rules directly from spiking data (Stevenson & Koerding, 2011; Robinson et al., 2016; Linderman et al., 2014; Wei & Stevenson, 2021) . Due to the gradient-based nature of our optimization technique, our proposed approach can only account for such data by converting spike trains to a rate-based representation by smoothing.Alternatively, meta-learning techniques (Thrun & Pratt, 2012 ) can be used to discover synaptic plasticity rules optimized for specific computational tasks (Tyulmankov et al., 2022; Najarro & Risi, 2020; Confavreux et al., 2020; Bengio et al., 1990) . The plasticity rules are represented as parameterized functions of pre-and post-synaptic activity and optimized through gradient descent or evolutionary algorithms to produce a desired network output. However, the task may not be welldefined in biological scenarios, and the network's computation may not be known a priori. Our method obviates the need for specifying the task, directly inferring plasticity rules from recorded neural activity or behavioral trajectories.Finally, Nayebi et al. (2020) do not fit parameters of a learning rule at all, but use a classifier to distinguish among four classes of learning rules based on various statistics (e.g. mean, variance) of network observables (e.g. activities, weights). Similarly, Portes et al. (2022) propose a metric for distinguishing between supervised and reinforcement learning algorithms based on changes in neural activity flow fields in a recurrent neural network. 
method overview: 
 Our goal is to infer the synaptic plasticity function by examining neural activity or behavioral trajectories in a learning organism. Specifically, we aim to find a function that prescribes changes in synaptic weights based on relevant biological variables. For simplicity, we consider plasticity localized to a single layer of a neural networkEQUATIONwhere x x x is the input to the plastic layer (the stimulus), y y y is the resulting output activity, and t is time. The synaptic weight matrix W t is updated at each time step based on a biologically plausible plasticity function g θ , i.e.:EQUATIONwhere θ parameterizes the function, x t j is the presynaptic neural activity, y t i the postsynaptic activity, w t ij the current synaptic weight between neurons i and j, and r t a global reward signal that influences all synaptic connections. We further define a readout function f that models observable variables M = (m m m 1 , . . . , m m m T ) of the network output y y y, i.e., m m m t = f (y y y t ). Those readouts model experimentalobservations O = (o o o 1 , . . . , o o o T ).In the context of neural activity fitting, the readout is a subset of y y y, whereas for behavioral models the readout aggregates y y y to yield the probability of a specific action. We will introduce our specific choices for readout functions in the following sections.The loss function L aggregates the differences (defined by a function ℓ) between the model's readout M and the experimentally observed output O across all time steps, i.e.:EQUATIONWe can now use gradient descent to find plasticity parameters θ, such that the loss between the model readout and observed values is minimized:EQUATIONIn experimental contexts, the observed behavioral or neural trajectories (O) can span extensive time scales, often consisting of thousands of data points. Computing gradients over such long trajectories is computationally demanding. Our framework is implemented in JAX (Bradbury et al., 2018) and designed to accommodate plasticity modeling across millions of synapses, anticipating future integration with large-scale connectomics datasets. Performance benchmarks reveal that our framework is capable of executing simulations with up to 10 6 synapses and 1000-time step trajectories in less than an hour on an NVIDIA A100 GPU. 
inferring plasticity from measured neural activity: 
 To illustrate and test this approach on neural activity, we generate a synthetic trajectory of observed outputs, denoted by O, using Oja's rule for synaptic plasticity (Oja, 1982) . The update rule for Oja's plasticity rule is given as follows:EQUATIONIn our evaluation, we use Oja's rule as a known a priori standard to validate the accuracy of our model in inferring synaptic plasticity (Figure 2A ). Since Oja's rule does not involve reward, we did not include a reward signal in this benchmark experiment. Inspired by previous work (Confavreux et al., 2020) , we parameterize the plasticity function g θ through a truncated Taylor series expansion,EQUATIONHere, the coefficients θ α,β,γ represent the model's learned parameters for the Taylor series expansion and were initialized i.i.d. from a normal distribution of mean 0 and variance 10 -4 . It is worth noting that Oja's rule can be represented within this family of plasticity functions by setting θ 110 = 1 and θ 021 = -1. For the following experiments, we set the readout function f to be an activity readout of a subset of all neurons, i.e., M = f (y y y) = (y i |i ∈ S), where S ⊆ [1, . . . , n], and n is the number of output neurons. We compute the loss as the Mean Squared Error (MSE) between the neural trajectories produced by both the Oja's rule-governed network and the Taylor series-based network,L MSE (M, O; θ) = 1 T T t=1 ||o o o t -m m m t || 2 . (7) 4.1 RECOVERING OJA'S RULEIn our experimental setup, we use a single-layer neural network with 100 input neurons and 1000 output neurons to generate synthetic data with a known ground-truth plasticity rule. The output layer uses a sigmoid activation function for its differentiability and biologically plausible output range of [0,1]. The readout selects a subset of neurons. The network is simulated over 50 time steps, during which we calculate the neural activity of all output neurons at each time step. For numerical stability, we apply gradient clipping at 0.2 to circumvent exploding gradient issues. Despite fitting on neuronal activity, Figure 2B shows a clear decrease in the error of the synaptic weight trajectories (horizontal axis) over training epochs (vertical axis). Figure 2C shows that the training dynamics for the plasticity coefficients leads to a correct recovery of Oja's rule.To evaluate the robustness of our method, we assess how varying degrees of sparsity in recorded neural activity and additive noise affect the model's performance. We use the R 2 metric to quantify the explained variance between the model-generated and observed weight trajectories post-training.Figure 2D illustrates the model's fit quality with increasing Gaussian noise levels and sparsity in the neuronal data. To simulate varying sparsity levels, denoted by a, we compute the loss using a fraction a of the neurons from the observed trajectories. The remaining (1 -a) fraction of neurons are modeled by incorporating random noise drawn from the same distribution as the observed neurons. Figure 2E delves into the noise sensitivity by exploring how different noise variances in the observed trajectories affect model performance. Figure 2F shows that our model retains a high degree of accuracy even when data from only 50% of the neurons is available, which is particularly advantageous given that sparse neural recordings are typical in experimental settings.  
inferring plasticity from measured behavior: 
 In extending our model's capacity for inferring synaptic plasticity, we apply it to observed simulated behavioral data. In controlled experiments, animals are frequently presented with a series of stimuli, with their choices to accept or reject these stimuli resulting in rewards and subsequent synaptic changes. Here we model this plastic layer of neural connections and the resulting learned behavior. The output neurons of the model network generate probability scores, subsequently read out to sample a binary decision to either "accept" or "reject" a presented stimulus. Upon accepting a stimulus, a stochastic reward is administered, which then alters the plastic weights in the network.The behavioral data available for training contains only these binary decisions. The model's synaptic weights are initialized randomly, mirroring real-world biological systems where initial synaptic configurations are typically unknown a priori. Our experimental setup is inspired by recent studies that have successfully mapped reward-based learning rules to observed behaviors in flies (Aso & Rubin, 2016; Rajagopalan et al., 2023) . Furthermore, existing research indicates that the difference between received and expected rewards is instrumental in mediating synaptic plasticity (Schultz et al., 1997; Rajagopalan et al., 2023) , and learning and forgetting happen on comparable timescales (Aso & Rubin, 2016) .For the synaptic updates, we therefore employ a reward-based plasticity rule (Figure 3A ). The change in synaptic weight ∆w ij is determined by the presynaptic input x j , the existing synaptic weight w, and a globally shared reward signal r. We neglect hypothetical dependencies on y i because they are known to not impact reward learning in flies. We primarily emphasize results from simulating a covariance-based learning rule (Loewenstein & Seung, 2006 ) that was inferred from previous experiments (Rajagopalan et al., 2023) . Here the reward signal is typically the deviation of the actual reward R from its expected value E[R], calculated as a moving average over the last 10EQUATIONHowever, we also show that our method is also able to recover several biologically plausible alternatives.For the model network, we employ a plasticity function parameterized through either a truncated Taylor series or a multilayer perceptron, i.e.,EQUATIONIn this behavioral paradigm, we utilize Binary Cross-Entropy (BCE) as the loss function to measure the discrepancy between the observed decisions and the model's output probabilities for accepting a stimulus:EQUATION)Note that this is also proportional to the negative log-likelihood function of the model. Parameters were initialized i.i.d. from a normal distribution of mean 0 and variance 10 -4 . The neural network model consists of three layers: an input layer with 100 neurons, an output layer containing 1000 neurons, and a single readout neuron that pools the output neurons to produce the probability of accepting a given stimulus. Upon choosing a stimulus, a probabilistic binary reward is delivered based on the choice. To encourage weight dynamics, the reward contingencies change every 80 trials. Given the discrete nature of the observed behavior and the continuous output of the model, we employ the percent deviance explained as a performance metric. The percent deviance explained measures the model's ability to account for the variability in observed binary choices compared to a null model that assumes no plasticity. It represents the reduction in deviance relative to this null model, expressed as a percentage. Higher values are directly associated with greater log-likelihoods, signifying a superior fit to the observed data.Figure 3B presents the weight dynamics of three models: a multilayer perceptron (MLP), a Taylor series approximation, and the ground-truth synaptic update mechanism. In the case of the MLP, we tested various architectures and highlight results for a 3-10-1 neuron topology. Our evaluation metrics-high R 2 values for both synaptic weights and neural activity-affirm the robustness of our models in capturing the observed data (Figure 3C ). The method accurately discerns the plasticity coefficient of the ground truth rule (Figures 3D, E ), albeit with a reduced magnitude. The model also does a good job at explaining the observed behavior (Figure 3F ).Additionally, Table 1 summarizes the recoverability of various reward-based plasticity rules for both MLP and Taylor series frameworks, with results averaged over 10 random seeds. It's important to note that solely reward-based rules (without E[R] or w) are strictly potentiating, as they lack the capacity for bidirectional plasticity. This unidirectional potentiation ultimately results in the saturation of the sigmoidal non-linearity. Therefore, it is possible to simultaneously observe high R 2 values for neural activities with low R 2 values for weight trajectories. In extending our model to biological data, we explore its applicability to decision-making behavior in the fruit fly, Drosophila melanogaster. Recent research (Rajagopalan et al., 2023) employed logistic regression to infer learning rules governing synaptic plasticity in the mushroom body, the fly's neural center for learning and memory. However, logistic regression cannot be used to infer plasticity rules that incorporate recurrent temporal dependencies, such as those that depend on current synaptic weights. Our method offers a more general approach. Specifically, we apply our model to behavioral data obtained from flies engaged in a two-alternative choice task, as outlined in Figure 4A . This allows us to investigate two key questions concerning the influence of synaptic weight on the plasticity rules governing the mushroom body. 
experimental setup and details: 
 In the experimental setup, individual flies were placed in a symmetrical Y-arena where they are presented with a choice between two odor cues (Rajagopalan et al., 2023) . Each trial started with the fly in an arm filled with clean air (Fig. 4A , left). The remaining two arms were randomly filled with two different odors, and the fly was free to navigate between the three arms. When the fly enters the 'reward zone' at the end of an odorized arm, a choice was considered to have been made (Fig. 4A , right). Rewards were then dispensed probabilistically, based on the odor chosen. For model fitting, we used data from 18 flies, each subjected to a protocol that mirrors the trial and block structures in the simulated experiments presented previously. Over time, flies consistently showed a preference for the odor associated with a higher probability of reward, and this preference adapted to changes in the relative value of the options (Fig. 4B ; example fly). Existing behavioral studies in fruit flies have shown that these insects can forget learned associations between stimuli and rewards over time (Shuai et al., 2015; Aso & Rubin, 2016; Berry et al., 2018; Gkanias et al., 2022) . One prevailing hypothesis attributes this forgetting to homeostatic adjustments in synaptic strength within the mushroom body (Davis & Zhong, 2017; Davis, 2023) . However, earlier statistical approaches aimed at estimating the underlying synaptic plasticity rule present in the mushroom body were unable to account for recurrent dependencies such as synapse strength (Rajagopalan et al., 2023) . Here we explore two types of plasticity rules: one based solely on reward and presynaptic activity, and another that also incorporates a term dependent on current synaptic weight. Both rule types allocate significant positive weights to a term representing the product of presynaptic activity and reward (Fig. 4C et al. (2023) . Intriguingly, our analysis additionally reveals that the inferred learning rule assigns a negative value to the weight-dependent term (Fig. 4C ). This negative sign aligns with the hypothesis that a weight-dependent decay mechanism operates at these synapses. The model with the weightdependent term also shows a greater coefficient for the purely presynaptic component (Fig. 4C ).THAN JUST REWARD?The previous study used reward expectations to generate bidirectional synaptic plasticity (Rajagopalan et al., 2023) . Our discovery of a negative weight-dependent component in the plasticity rule provides an alternate mechanism for this, raising the question of whether the neural circuit really needs to calculate reward expectation. Could a plasticity rule incorporating the product of presynaptic activity and absolute reward combine with a weight-dependent homeostatic term to approximate a plasticity rule that involves reward expectation? To answer this, we contrast two models: one using only the absolute reward and another using reward adjusted by its expectation, both complemented by weight-dependent terms. Our analyses show that adding a weight-dependent term enhances the predictive power of both models (Fig 4E, F) . However, the model that also factors in reward expectations provides a superior fit for the majority of flies in the data set (Wilcoxon signed-rank test: p = 0.067; Fig 4F ). These compelling preliminary findings reaffirm the utility of reward expectations for fly learning, and larger behavioral datasets could increase the statistical significance of the trend. Overall, our model-based inference approach, when applied to fly choice behavior, suggests that synaptic plasticity rules in the mushroom body of fruit flies are more intricate than previously understood. These insights could potentially inspire further experimental work to confirm the roles of weight-dependent homeostatic plasticity and reward expectation in shaping learning rules. 
limitations and future work: 
 Despite its strengths, our model has several limitations that offer avenues for future research. One such limitation is the lack of temporal dependencies in synaptic plasticity, neglecting biological phenomena like metaplasticity (Abraham, 2008) . Extending our model to account for such temporal dynamics would increase its biological fidelity. Another issue is the model's "sloppiness" in the solution space; it fails to identify a unique, sparse solution even with extensive data. As neural recording technologies like Neuropixels (Steinmetz et al., 2021; 2018) and whole-brain imaging (Vanwalleghem et al., 2018) become more advanced, and connectome data for various organisms become increasingly available (Bentley et al., 2016; Scheffer et al., 2020; Hildebrand et al., 2017) , there are exciting opportunities for validating and refining our approach. Incorporating these highresolution, large-scale datasets into our model is a crucial next step. In particular, future work could focus on scaling our approach to work with large-scale neural recordings and connectomics, offering insights into the spatial organization of plasticity mechanisms. Additional considerations for future research include the challenges posed by unknown initial synaptic weights, the potential necessity for exact connectome information, and the adequacy of available behavioral data for model fitting.https://anonymous.4open.science/r/MetaLearnPlasticity-A3E3/ 
 
 Appendix:  


Decision: The paper proposes a nice approach for inferring synaptic plasticity rules from (partially) observed neural activity or behavioral read-outs. The approach is fairly straightforward, but it seems to work well across several examples. I found it surprising and impressive that behavior in a binary decision making task could say much about the nature of the plasticity rule. 

The reviewers raised some very good points, and I encourage the authors to incorporate their feedback and make the promised changes.

Reviews: review: paper_summary: paper_summary: This paper presents a reward learning-based method for recovering biological synaptic plasticity rules in a model network. It is meant to be applied to both neural and behavioral data. The method consists of taking real learning trajectories collected in response to stimuli (of some observable metric modality) and showing those stimuli to the model artificial neural network, then computing the loss between the generated and. modeled trajectories and taking a gradient step in the direction of the ground truth data. The weight update is determined by some plasticity function, represented by either a Taylor expansion or an MLP, and also learned over the course of training according to the trajectory loss. After training the model network, various plasticity rules are tested by analyzing the learned parameter values of the plasticity function. 

After establishing the method, the paper presents specific experimental settings and results. It begins with a simpler case of simulated neural activation data with plasticity dynamics based on Oja's rule, MSE loss between the two trajectories, and a Taylor series plasticity function. The paper shows that Oja's rule can be recovered - it is seen clearly in the learned parameters of the Taylor series plasticity function. Ablations are also conducted over increasing sparsity and noise, showing degradation. 

The paper then goes on to a simulated behavioral data setting with a small MLP representing the plasticity function. Here, it establishes percent deviance explained between the ground truth and modeled trajectories as a performance metric. Finally, it demonstrates some proof of concept in actual drosophila behavioral data, including a forgetting mechanism (negative dependency on the weights, not just the reward signal and presynaptic activations). 

Finally, it goes through related work.
strength_weakness: strength_weakness: ### Clarity
- Paper is very jargon-heavy without defining niche terms and having clear conclusions in paragraphs. It would help to have simpler language and/or more explanation
- Percent deviance explained - we don't get grounding or baselines to understand how to judge these results. Are there baseline methods, or any kind of comparison point/context?

### Quality
- It's not clear how far this method, or even the principles established in the paper, can take us. 
  - The space of tested plasticity rules seems hand-designed and limited. Rules are rejected only when they can be clearly defined and tested. 
  - Percent deviance explained results seem to leave plenty of room for improvement 
  - Test setups are quite simple. This is mitigated highly by the presence of real drosophila behavior data - that experiment is really useful - but still, simple and limited, particularly in the underlying dynamics 
  - The paper argues that it is "reasonable" to assume that we can have the modeling network architecture exactly match the true architecture, because of available connectome data. Connectomes are static and extremely detailed, and there are lots of elements (e.g. immense recurrence) that we do not have architectures for. This is again very much improved by the real drosophila data experiment, but it is nevertheless concerning. 
  - Underlying plasticity rules in synthetic settings are very simple, even in the case being learned by an MLP. The drosophila behavioral experiment helps because we see the discovery of the forgetting rule, but we don't have any sense of a full space to look for, or even a partial but extensive space to look for (things that both should and shouldn't show up). Even the "forgetting" rule discovery is believable but requires a lot of interpretation of results. 
- Ablations aren't really discussed. Noise and sparsity make sense as ablation factors, but what do the results mean beyond just "noise and sparsity cause problems"? Especially because their effects are strangely similar.
questions: questions: - How should I ground my judgement of the percent deviance explained results?
limitations: limitations: Limitations - yes
Societal impact - they include the NeurIPS checklist in their supplementary and say the societal impacts have been addressed, but they haven't anywhere, nor is there an assertion that they don't need to. They probably don't, though.
paper_summary: paper_summary: This paper studies how to learn local learning rules (like those plasticity rules thought to be used by real neurons) in a data-driven way from neural activity or behavioral timeseries. This is applied to simulated as well as fly behavioral data. The true learning rules can be learned from synthetic activity timeseries when complete observations are available (Fig 2), whereas from behavior alone some more error is incurred (Fig 3). When using fly behavioral data, the authors get better fits than previous methods (Fig 4), and a claim is made that the "forgetting term" in the resulting fit is important.
strength_weakness: strength_weakness: I think the main weakness is what I identified below in limitations: the degeneracy or non-identifiability of such models. I can imagine that even in settings where the equations are the correct model that it is not identifiable without some regularization.

The model fits to behavioral data are given as R^2 values on the behavior themselves. It does not seem that any cross-validation was used for model fitting, so there is a possibility that your model is overfitting. This should be discussed.

I'll admit I'm not super familiar with existing work in this area, so I am not sure how novel these results are relative to what's been tried before.
questions: questions: * What justification do you have for claiming that your model requires less energy than an LLM of the same size? (Line 45-46)
* Can you be precise about which term in Fig 4C is the decay term? I think you mean 000. It isn't evident whether that is significantly different from 0. The error bars are quite wide.
* Have the authors considered the vast literature on fitting dynamics themselves from timeseries? I am familiar with the SINDy method, which uses polynomial or other families of basis functions and includes a LASSO-type penalty to fit dynamics. It is likely that some regularization or penalty could improve the learned models in your setting, too. (I now see this buried in the appendix, but it should be discussed in the main text.)
* Please discuss why you did or didn't use cross-validation
limitations: limitations: I think it's a stretch to claim that the underlying connectivity (presence of nonzeros in weight matrix) is known because of connectomics (lines 80-82). That may be true for some small model organisms like Drosophila, and only just, but it is nowhere near true for others such as the mouse. The authors should be careful about making claims of identifiability with models such as theirs. For instance, it's known that incomplete knowledge of the connectome or partial observations of dynamics can lead to spurious correlations. An example that comes to mind is https://journals.aps.org/pre/abstract/10.1103/PhysRevE.109.044404 .

Your experiments actually show this non-identifiability already: In figure 2G we see that the Oja rule is not recovered when the observations are incomplete. There is a brief discussion of this "degeneracy" in the conclusions and limitations section, but I think it is a bigger issue than the authors are claiming. In particular, this conjecture that "in the infinite data limit the fitted plasticity rules would, in fact, be unique" could easily be wrong. It is certainly not a useful argument to the experimentalist who will always have limited data, especially if it's behavioral or brain recording data.
paper_summary: paper_summary: This paper proposes a novel method for inferring plasticity rules from neural and behavioral data. In contrast to previous approaches, the plasticity rule is directly optimized to maximize the similarity of the output of a model trained with the plasticity rule to a target (neural activity or behavior). This approach is validated with synthetic experiments and used to infer plasticity rules from fruit fly behavioral data.
strength_weakness: strength_weakness: * My sole concern with this method is its broader applicability to more complex problems; inferring the learning dynamics involved in a simple forced choice task where the biological network is relatively well-known is very different from partially observed neural activity with poorly-understood connectivity during free behavior. I'm particularly concerned about the case where the influence of a large number of unobserved neurons needs to be considered.
questions: questions: * Double-check the subscripts in Eq (5), I think $y_j$ should be $y_i$.
* For the bottom portion of Fig 2, are the weights being compared the weights after training when using the ground truth and inferred rules?
* Is there some explanation for why the last rule in Table 1 exhibits such different behavior?
* Is there held out validation data in the experiments in Section 5, or is a single dataset used throughout?
limitations: limitations: Yes

Comments: title: Increasing score to 7
comment: Thanks for the detailed comments. The main thing imo is that you've successfully argued that the limitations of the paper are more reasonable and field-standard than I was aware of. Increasing my score to a 7.
Thanks for your responses. I think that if you address these comments in sufficient detail in the final version, that is sufficient. I am going to adjust my overall score to a 7 and soundness score to a 3.
I appreciate the thoughtful response. I maintain that this is an interesting paper that proposes a clean and fairly general method to an important problem and recommend it for acceptance on that basis alone. I agree with the other reviewers that the potential for broader impact is less clear. There seem to be good reasons to doubt that this method or its underlying principles will scale to more complex paradigms, but this is ultimately an empirical question which will only be answered by substantially more involved experiments in follow-up work. I stand by my original score.