Flexible inference for animal learning rules using neural networks
Authors: Yuhan Helena Liu
Victor Geadah
Jonathan W. Pillow
Venue: NeurIPS 2025 poster ()
Keywords: Computational neuroscience
learning rule inference
decision-making
de novo learning
neural networks
animal behavior
Primary area: neuroscience_and_cognitive_science

Abstract: Understanding how animals learn is a central challenge in neuroscience, with growing relevance to the development of animal- or human-aligned artificial intelligence. However, existing approaches tend to assume fixed parametric forms for the learning rule (e.g., Q-learning, policy gradient), which may not accurately describe the complex forms of learning employed by animals in realistic settings. Here we address this gap by developing a framework to infer learning rules directly from behavioral data collected during *de novo* task learning. We assume that animals follow a decision policy parameterized by a generalized linear model (GLM), and we model their learning rule—the mapping from task covariates to per-trial weight updates—using a deep neural network (DNN). This formulation allows flexible, data-driven inference of learning rules while maintaining an interpretable form of the decision policy itself. To capture more complex learning dynamics, we introduce a recurrent neural network (RNN) variant that relaxes the Markovian assumption that learning depends solely on covariates of the current trial, allowing for learning rules that integrate information over multiple trials. Simulations demonstrate that the framework can recover ground-truth learning rules. We applied our DNN and RNN-based methods to a large behavioral dataset from mice learning to perform a sensory decision-making task and found that they outperformed traditional RL learning rules at predicting the learning trajectories of held-out mice. The inferred learning rules exhibited reward-history–dependent learning dynamics, with larger updates following sequences of rewarded trials. Overall, these methods provide a flexible framework for inferring learning rules from behavioral data in *de novo* learning tasks, setting the stage for improved animal training protocols and the development of behavioral digital twins.

Flexible inference for animal learning rules using neural networks
Yuhan Helena Liu"*, Victor Geadah', and Jonathan Pillow
Princeton University, Princeton, NJ, USA "Correspondence: hi7582@ peeee@@rrriinnneiii edu, pillow princeton.edu
Abstract
Understanding how animals learn is a central challenge in neuroscience, with growing relevance the development of animal- or human-aligned artificial in- telligence. However, existing approaches tend to assume fixed parametric forms for the learning rule (e.g., Q-learning, policy gradient), which may not accurately describe the complex forms of learning employed by animals realistic settings. Here address this gap by developing framework to infer learning rules di- rectly from behavioral data collected during de novo task learning. We assume that animals follow decision policy parameterized by generalized linear model (GLM), and we model their learning rule-the mapping from task covariates to per-trial weight updates-using a deep neural network (DNN). This formulation allows flexible, data-driven inference of learning rules while maintaining an in- terpretable form of the decision policy itself. To capture more complex learning dynamics, we introduce a recurrent neural network (RNN) variant that relaxes the Markovian assumption that learning depends solely on covariates of the current trial, allowing for learning rules that integrate information over multiple trials. Simulations demonstrate that the framework can recover ground-truth learning rules. We applied our DNN and RNN-based methods to a large behavioral dataset from mice learning to perform sensory decision- makion task and found that they outperformed traditional RL learning rules at predicting the learning trajectories held-out mice. The inferred learning rules exhibited reward- hhitory-dependee learning dynamics, with larger updates following sequences of rewarded trials. Overall, these methods provide flexible framework for inferring learning rules from behavioral data in de novo learning tasks, setting the stage for improved animal training protocols and the development of behavioral digital twins.
Introduction
The study of animal learning has long history, beginning with foundational work such as Pavlov's classical conditioning and the Rescorla- Wagner model of associative learning 2 Uncovering the mechanisms that underlie learning and decision-making is crucial for understanding and predicting animal behavior (3. This knowledge has wide-reaching applications, including benefits for human health and medicine 4+7 informing conservation efforts by anticipating how animals respond to environmental change [2 and potentially guiding the development of biologically aligned AI
systems, which have garnered increasing interest in recent years 13 14
While inference of learning rules is gaining traction, prior studies have key limitations. For example, many leverage reinforcement learning (RL) paradigms with normative approaches that assume reward maximization 17 or weight update under a parametric learning rule (e.g.. Q-learning [8]. Although these approaches have provided clear insights into animal learning,
REINFORCE 19 20
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

they may lack the flexibility to capture the detailed structure of real learning rules, which are often unknown and may not align with parametric forms conceived by human researchers. As a result, existing models may span only a small subset of the space of possible learning strategies animals actually use. Recent work has proposed more flexible quantitative models of learning based on recurrett neural networks (RNNs) [21 or mechanisms proposed by large language models [25]. However, these studies have generally focused on "bandit" style tasks, in which animals adapt their behavior to time-varying rewards in fixed task structure, opposed to learning novel task or behavior from scratch 126 27]. De novo learning is common in experimental neuroscience, as animals are routinely trained to acquire novel input-output mappings from scratch, yet comparatively understudied. Finally, a variety of descriptive models have been developed to characterize trial-by- trial changes in weights that govemn an animal's policy as it evolves over learning, but do not identify any rules governing these weight updates [28
To address these shortcomings, we propose new approach with the following main contributions. To our knowledge, this the first approach to infer nonparametric, non-Markovian learning rules directly from behavior in de novo tasks.
1. Inferring flexible learning rules using DNNs. We develop a deep neural network (DNN) modeling approach to infer flexible animal learning rules from de novo task learning data (Figs. 1 Tand
2. Non-Markovian learning rules. Many classic learning rules are Markovian "memory- less", meaning that the learning update depends only on the input, action, and reward on the current time step. Such rules, including own DNN approach, thus fail to capture "non-Markovian" learning, where updates depend on multiple time steps of trial history. To overcome this limitation, we incorporate recurrent neural network (RNN) capture arbitrary dependencies of learning on trial history (Fig. 3
3. Improved held-out prediction on real data. We apply our methods to mouse training data from the International Brain Laboratory (IBL) and observe significantly higher log- likelihood on held-out data compared to existing approaches (Fig.
4. Insights into animal learning strategies. We analyze the inferred learning rules and identify several departures from classic policy gradient learning, including non-Markovian reward history dependencies lasting multiple trials (Fig. 5)
2 Model and methods
Model of decision-making. We begin with a dynamic Bernoulli generalized linear model (GLM) of an animal's 's time-varying behavior during learning [19 20 model of an animal's 's decision- decision-making policy in terms of set of weights that evolve dynamically over the course of learning. This differs from recent work in which the animal's 's policy is directly
28 30) This provides interpretable
parametrized by an RNN 21 22 which offers increased flexibility but may be challenging to interpret.
On trial f, a sensory stimulus 8₁ E [-1, appears on the left or right side of the sereen, where s; > 0 indicates the fractional contrast of a right-side stimulus and s; 0 indicates the (negative) fractional contrast of a left-side stimulus. The mouse makes a binary choice y/s E {0, 1} by turning a wheel, where 0 corresponds to leftward and 1 to rightward choice, respectively (Fig. (A). The true label of the stimulus is denoted as 2 = 1[s > 0], representing the correct response according to the task.
We assume the animal's policy on trial is governed by weight vector w, E Rdd which describes how task covariates, denoted x, E Rd4 influence the animal's choice. Unless otherwise specified, we will define the input vector to be x = [st. 1] consisting of the signed stimulus intensity s; and a constant or "bias" term [30). This input interacts linearly with the weight vector w,, determining the probability that the animal selects the "rightward" choice on trial t:
ply X we) 1te
(I)
The reward function given by r. = 1 1y = =x]. which corresponds to positive reward of 1 if the animal makes correct decision (y/² 21) and reward of 0 otherwise.

---

A
de novo task learning
DNN-based learning rule (Markovian learning)
incorrect
performance
Awr
fo( We, yt,
DNN
trial #
Aw
(IBL 2020)
3/
Tt
B
D RNN-based learning rule (non-Markovian)
trial
stimulus
St-1
8t
h go ( 8r, We, We,
W -
Wz
Aw, so(he)
policy weights
Yt
W
W
rt-1
Ft
DNN
learning
A W
response
M/+
5t++1
Aw:
hidden
reward
72 T++1
state
x
h
he
Figure 1: Task schematic and learning rule inference methods. (A) We examine learning of a sensory decision-making task, in which mice must learn report which side of the screen contains visual stimulus by turning a wheel [3I. (B) In our framework, decision-making is governed by the weights of Bernoulli generalized linear model (GLM), which evolve across trials according an unknown learning rule. (C-D) To infer the learning rule, we approximate the weight update function Aw; using either (C) a deep neural network (DNN) that maps the current trial covariates weight change: or (D) recurrent neural network (RNN) that integrates information from previous trials before feeding into the DNN. We optimized the neural network model parameters by maximizing the log- reg-oobbabiliyy of the animal choice data under the dynamic Bernoulli GLM (see Methods and Algorithm D. We refer to our approaches as DNNGLM or RNNGLM depending on the model used to parametrize the learning rule.
DNNs for learning rule inference. Learning in this paradigm is captured by updates to the decision weights w. We first model the weight update function using feedforward neural network (Fig. [C]:
Aw; fo Wr, Xt, Dt. r'):
where fa is a feedforward network with two hidden layers and trainable parameters 0 (see Supp. C for architecture details). The next weight is given by:
W++1 W. AW
Initial weights Wo be treated as trainable parameters, initialized via psychometric curve estimates (Supp.C) C).
The inferred weights w, parameterize the GLM response distribution (Eq. [. and network parameters @ are optimized maximize the log- likelihood of observed choices:
(e xx) = log p(ye = Xx, w.) + (1 - -y)) log(1 - p(y: = 1 xr, w.)).
which corresponds minimizing binary cross-entropy loss.
RNNs for learning rule inference. Since feedforward networks are memoryless, we also consider a recurrent parameterization using GRU 32] to encode history. The same inputs {wr, Xt, Mr, rr) are passed into a GRU go to produce a hidden state h,, which is then passed to the feedforward network:
h go (h1-1, Wt. X1, Tz),
(5)
Aw; = (h)).
(6)
The model is trained using the same objective and procedure above (see also Supp.
By combining GLM for choice modeling with neural network-based learning rules, we balance interpretability and flexibility. We refer to the two models as DNNGLM and RNNGLM. depending
3

---

on whether the learning rule is parameterized with a feedforward or recurrent network. Cross validation, together with the implicit regularization properties neural networks, helps mitigate overfitting; further details regarding our model and data processing can be found in Supp. The full procedure outlined in Algorithm
Algorithm 1 Learning Rule Inference from Trial-by-Trial Behavior
1: Inputs: Trial sequence {st, l/t, Te)
2: for trial 1 to 7" do
3: Construct input vector: = [st, Ve. rz. we]
4: Compute weight updates: Aw, < NN((u)
5: Update the inferred GLM weights: Wi+1 + We + Aw.
6: Predict choice probability: Py, = o(w, x²))
7: Accumulate binary cross-entropy loss: Lt
[ye log Pe. + (1 ya) log(1 - Pse)) )
8: end for
9: Objective: Minimize total loss L = Ei= L, using gradient descent
Simulation results: recovery of ground-truth learning rules
A Markovian learning rule: (Wi, St, Me. r2) Aw:
C
2x10
Points R==0.988
fitted
Wstmm
true
RMSE
Trials
10:
10
10
Number samples
B
true, after correct
true
fitted
after correct
Wstim
AWstim
after incorrect
w=-1
AWstim
0.1-
0.1
W=1
0.0
-2
Stimulus
Stimulus S
Stimulus
Figure 2: Recovering ground-truth learning rules in simulated data. We simulated animal learning using a policy gradient method known as "REINFORCE" [33]. (See Supp. Fig. F alternative ground-truth learning rule). This rule is Markovian in that the weight update Aw, depends only on current-tri trial variables. (A) Simulated trajectory for stimulus weight Watsm for an example simulated mouse (green) and the inferred weights using the DNNGLM (blue). (B) Left: Heatmap showing the ground-truth stimulus weight change Awstim following a correct choice, as a function of Watim and stimulus Horizontal dashed lines indicate Wstim right panels. Middle: slices showing stimulus weight change AW as a function the stimulus 8, for different values of the true weight Watim. after both correct (solid) and incorrect (dashed) decisions. (The REINFORCE algorithm exhibits no learning after incorrect trials for this setting of rule parameters). Right: corresponding slices through the weight update function inferred using the DNNGLM. Note that the model successfully captures key characteristics of the learning rule, such the slowing of learning at higher weights, increased learning with stimulus amplitude. and the asymmetry in learning after correct and incorrect choices. (See Supp. Fig. [orr analogous plots for bias parameter updates and Supp. Fig. for comparison other learning rules.) (C) Error in recovered learning rule function of dataset size, indicating that the DNNGLM converges to the true REINFORCE learning rule with increasing amounts of data.
for an
slices shown in middle and

---

Non-Markovian learning rule: (..., 1y 8t- 1,-1; Wr, A1; Aw:
true
DNNGLM
RNNGLM
Wetim after correct y
0, correcty
after correct
0.5
0.5-
0.5
conditioned on
past 3 trials rewarded
AWstim
AWWattm
AWstim
0.0
0.0-
0.0-
conditioned on past trials unrewarded
2
5 Slimulus
2
-2
-1
-2
0
Stimulus
Stimulus
Figure 3: Simulated example with non-Markovian learning rule. The RNNGLM recovered the reward history dependence present in the ground truth: larger weight updates occur when the past 3 trials were rewarded (black) versus unrewarded (red). Such differences were absent in DNNGLM, as it can only account for Markovian learning rules, where weight updates depend on current trial information. The ability of RNNGLM capture these effects reflects its greater flexibility. Minor deviations from the ground truth are likely due to finite data effects (Fig. (c). To keep the figure simple, we only plotted with the learning rule given stimulus weight Wstim 0 and after correct choices, but we observed similar phenomena for other parameter settings.
To begin our investigation, we first consider simulated data where the ground truth is known. We consider two types of learning rules: Markovian and non-Markovian. By Markovian learning rules, we mean that weight updates depend only current-trial variables and are thus conditionally independent of stimuli, choices, and rewards on previous trials. contrast, non-Markovian learning rules allow weight updates on the current trial exhibit arbitrary dependencies on task history.
For Markovian learning rules, we considered classic policy gradient learning rule known as REINFORCE 33 which is defined by:
Awe x rz V logp(ye Xe, W+)
Tx (1 Pve )Za,
6yx=R +1,
Ey:=L 11
where Py, == p(yc X4, we), the probability of the choice y/s given input x2 under the animal's 's current policy. REINFORCE is a policy gradient method that uses Monte Carlo integration to evaluate the gradient of the policy with respect to reward (details in Supp. B.T.
We first validated our framework on synthetic data where the ground truth learning rule is known (Fig. 2 When simulating animal learning under REINFORCE [33], our method (DNNGLM) recovered the weight trajectory and key properties of the learning rule, including dependencies on decision outcome, stimulus contrast, and current weight. Reconstruction accuracy- as the log base 10 of RMSE between the ground-truth A and the model-predicted Áw on the training data-also improves with increasing data (Fig. 2C).
We then simulated Mar-Markovinn learning rule in which weight updates depended on trial history (... Specifically, we consider modified REINFORCE rule with digibility-trace- factor derived from non-Markovian task setting (see Supp. B.2 for details):
Aw; x T Σ G c Py. Pv.., Xt-
A key property of this rule is that correct and rewarded past trial contributes nonnegative term to the sum - since ey.,, and X;-a share the same sign and the remaining terms are nonnegative while an incorrect trial contributes nonpositive term. Thus, conditioning rewarded (versus unrewarded) leads to larger weight updates under the ground truth learning rule. This dieeenetde effect is successfully captured by the fitted RNNGLM (Fig. B}.
several past trials being
In this case, the DNNGLM model fails to recover the correct learning dynamics; RNNGLM recovers the reward-histon dependence in the learning rule. This demonstrates the need for recurrent architec- tures in modeling non-Mard Markovinn,,,, dieenett learning processes. In principle, the weights could carry some history even in DNNs, since <- + Aw integrates past updates. However,
5

---

the learned update function Aw itself does not explicitly depend on past trial information and thus cannot capture
learning dynamics (see F).. While expanding the DNN imput to include historical information is possible, this would require manual feature engineering and reduce the model's ability to automatically determine which past inputs are relevant.
A
Example heldout animal
B
PsyTrack
Worse Relalike Log-L kalhhoo
DNNGLM
600
2000
4000
6OOD
BOCO
10000
Wbbs
REINFORCE
DNNGGL
DNNNLM
(notory)
RNNGUU
4000
6000
10000
Trials
Figure 4: Application to IBL mouse learning data. (A) Predicted stimulus (top) and bias (bottom) weight trajectories for a held-out animal - using DNNGLM trained on other animals and applied to this animal's 's stimulus and choice sequence PsyTrack [28]. which is a learning-agnenn agnostic and reliable method for tracking psychometric weights from behavioral time series. (Supp. Fig. [3 shows additional weight trajectories.) Similar traces were observed for RNNGLM as well. (B) Our methods (DNNGLM and RNNGLM) achieved significantly higher test log-likelihood (L.L) held-out data. Here we plot LL relative the REINFORCE model. Notably, the RNNGLM M model also out-performed the DNNGLM extended to include the previous trial input ("DNNGLM- history")). Error bars reflect standard deviation across validavaiio seeds.
plotted along the weight trajectories inferred by
We also add more results in the Supp. Material demonstrating that our main findings are robust across a variety of conditions:
(i) a learning rule without the decision outcome asymmetry (Supp. Fig. 6): (ii) recovery of an in-between learning rule when mixing update functions across individuals (Supp.
(iv) longer trial lengths (Supp. Fig. TOp. While the initial policy weights are typically unknown, we estimated them from the psychometric curve (Supp. C) and demonstrated recovery in Supplementary Fig. I0. For faster simulations in the main text, and to allow multiple repetitions for robustness testing, we used shorter trial sequences. These shorter sequences make it difficult to estimate the initial weight due to fewer samples at the beginning, so initialized with known initial weight in this case. Robustness tests showing recovery with incorrect Wo are provided in Supplementary Table We also show sensitivity to the initial weight estimate (Supp. Table 3) and degradation of reconstruction under added noise (Supp. Table
Fig. [T): (iii) different initial stimulus weights (Supp. Fig. 9
4 Application to mouse learning data
For real data, we used the publicly available International Brain Laboratory (IBL) dataset [3]] following the same preprocessing as [19. We first fit the learning rule on pool of training animals. We then took the stimulus and choice sequence from held-out test animal and applied the fitted DNNGLM or RNNGLM to predict the evolution of its stimulus and bias weights. We compared the predicted weight trajectories to those inferred by PsyTrack (28). a learning-agnose and reliable method for tracking psychometric weights from behavior, and observed close agreement (Fig. [A). Furthermore, using the predicted weights to compute the choice log-likelihood on held-out animals, both DNNGLM and RNNGLM outperformed REINFORCE
models (Fig. 4B and Table D. We also assessed temporal generalization on future held-out data and observed similar trends (Supp. D Notably, RNNGLM also outperformed history-augmenter DNNGLM, in which previous-trial information was added as an additional regressor to the GLM.
Moreover, our methods revealed empirical features of mouse learning that deviate from classical learning rules (Fig. (5)). These include the presence of a "negative baseline" (coined in 20 but also observed our flexible framework), where weight updates following incorrect trials actually decreased task accuracy even though a positive update would aid learning in this particular task. We also observed asymmetries in update magnitude based on decision correctness, with distinct updates following correct (solid lines) versus incorrect (dashed lines) choices, as well asymmetries

---

A
DNNGLM
RNNGLM
REINFORCE (fitted)
0.06
0.05
0.05
after correct y
W=0
WF1
AWstim
Wm2
AWstim
AWstim
0.00
0.00
0.00
incorrectty y
40.06
-0.05
0.0
-10
0.5
0.0
0.5
1.0
-1.0
0.5
0.0
0.5
Stimulus
Stimulus
Stimulus
S
B
Conditioned past 1 trial
on past trials
past 4 trials
past unrewarded, current correct
current correct
AWstim
AWsttm
AWstim
0.00
0.00
0.00
past unrewarded, current incormet
past rewarded, current incorrect
0.00
-0.05
-0.05
-1.0
Stimulus
Stimulus S
Stimulus 5
Figure 5: Properties of inferred learning rules. (A) For the dataset in Fig. DNNGLM reveals negative weight updates following errors (coined as 'negative baseline' in 20 which was also observed in flexible framework). Bias weight plot is provided in Supp. Fig. note is fixed positive weights because the training data mainly involved positive w. We follow the same plotting convention in Fig. 2 since RNNGLM depends additional historical variables, we plot the mean Atatim averaged across these historical dimensions. (B) RNNGLM suggests non-Markovianity (history dependence) in learning: larger weight updates are observed when all past trials were rewarded (black) versus unrewarded (red). This gap widens as we condition on history beyond just the previous trial, suggesting that the history dependency extends beyond the most recent past trial. We note that plotting up to four past trials was intended only as an illustrative example, not fundamental limit of RNNGLM; dependencies. To keep the figure simple, we only plotted with stimulus w fixed at O, but similar trends are observed elsewhere.
shown Supp. Fig. [3 the model suggests longer history
across stimulus sides (i.c., different update functions for left vs. right choices). Notably, such side asymmetries have also been reported in prior work (34).
In addition, the fitted RNNGLM - achieving the best held-out log likelihood showed higher updates following sequences of rewarded trials (Fig. [53), suggesting sensitivity to reward history spanning multiple trials (see also Supp. Fig. BT Together, these findings highlight the framework's ability to improve predictive accuracy while uncovering mechanistic insights into animal learning.
Because inferring flexible learning rules from binary choices is an under-constrained inverse problem, multiple parameterizations can fit the data similarly well. We therefore repeated inference for each model (DNNGLM, RNNGLM) across multiple random seeds, holding data splits and hyperparame- ters fixed. Across seeds, log-lik likeeliss were similar, while the recovered update functions showed modest variation. To visualize this, we now plot mean and standard deviation across weight updates across seeds (Supp. Fig. 12). Despite some variability. the key characteristics are consistent across seeds negative baseline, side asymmetry, and multi-trial (non-Matte Markovian) dependence on reward/stimulus history. Moreover, the elimination of these features suggests they are functionally meaningful: enforcing non-negatyv constraint on the baseline reduces held-out predictive perfor- mance (Table TT. Additionally, to assess fairness, we matched parameter counts between DNNGLM and RNNGII .M, and observed that RNNGLM still outperformed (p = 5.53e 4), indicating the gap is not due to just model size.
7

---

Comparison
Heldout LL
p-val
REINFORCE vs. DNNGLM
60576 vs. -60360
1.57e
DNNGLM vs. RNNGLM
60360 vs. -59892
1.79e 8
REINFORCE vs. nonnegative base
-60576 vs. -68532
1.25e
tinyRNN vs. RNNGLM
60564 vs. 59892
2.12e 3
REINFORCE (history) vs. RNNGLM
60504
59892
7.73e-7
DNNGLM (history) vs. RNNGLM
600168 vs.
59892
3.84e-0
Table 1: Animal-level held-out log- likelihoods (higher is better) for Fig. 4B, except that we report total log- log-likelihood (summed across animals) instead of relative log-likelihood. We show p-values from paired t-tests across mice, comparing different learning rule models on the IBL dataset. Each entry shows two values (one per model in the comparison), reported as mean + standard error across cross- validaaaii seeds. For the interpretable RNN baseline which predicts behavior directly rather than parameterizing weights or weight updates - we used tinyRNN [2] with 8 GRUs, more than the 1-4 GRUs used in their original paper, which performed even worse in our tests. Results for held-out future data also show consistent trends (Supp. Table
Related Work
Previous studies have shown that artificial neural networks provide more accurate description of animal learning behavior than classical learning models 52) Several recent works have taken steps to enhance the interpretability of inferred learning strategies for example, by regularizing the latent space or parameterizing in low-dimensional or guman-interpretable forms [21-23] 25], but they have not addressed de novo learning, i.e., learning perform a new sensory-motor task behavior structure (input-output mappings) from scratch.
Methods for inferring learning rules during the acquisition of new task structures remain sparse. Those that do exist typically assume a predefined functional form [18 20 34], limiting their ability to capture learning strategies that differ from hand-designed models. In addition, many memoryless or "Markovian', assuming that weight updates depend only on the task variables encountered on the current trial; this stands in contrast to biological learning, where history plays a prominent role such as through eligibility traces [53 54 or the influence of trial history on decision-making [30 [55) 577. In this work, we focus on inferring learning rules from behavioral data, motivated by the broad availability of behavioral datasets and the potential for large-scale inference without requiring simultaneous neural recordings. In parallel, several studies have proposed approaches to infer synaptic-leve plasticity rules (58)66), which offer detailed insights into neural mechanisms. While some of these approaches can also be applied behavioral data (e.g.. 621). their focus has largely been on neural-level inference and often requires neural activity recordings or handcrafted functional bases for interpretation. Our method instead emphasizes flexible inference directly from behavior, improving weight recovery via psychometric-cury initialization and extending flexibility with recurrent networks to capture multi-trial history beyond bandit tasks.
Beyond neuroscience, our work is also related to the meta- lettaleernnng literature. A prominent line of work learns optimizers or update rules via gradient-based meta-learning (67), with extensions to reinforcement learning [44 68); several computational neuroscience studies likewise meta-learn synaptic plasticity rules within interpretable frameworks 41] 63
70 In contrast to meta-learning approaches that optimize agents to learn well across tasks, we address a fundamentally different goal: inferring, from behavior alone and in de novo tasks, the learning rule that a biological agent used to learn from scratch. This poses distinct challenges: (i) animals often exhibit suboptimal or biased strategies that violate normative space and introducing external stimuli not just reward histories going beyond classic bandit settings; and (iii) many prior approaches fix feature representations, use static policy forms, or rely on hand-designed rules that do not adapt during training. These factors call for flexible yet interpretable models with minimal structural priors; under these conditions, prior baselines underperform relative to ours (Table Our approach thus helps bridge structured neuroscience models with black-box meta-learning in this de novo inference setting.
69
assumptions [57 7T], enlarging the solution and other idiosyncrasies; (ii) updates must integrate
8

---

6 Discussion
Understanding how animals learn from experience-particuuls in de novo settings where in- put-output mappings must be acquired from scratch-remaine central challenge. Prior work has largely relied on predefined parametric models or focused on simpler settings like bandit tasks that do require learning a new task structure from scratch. Here we instead infer, from behavior alone, the trial-by-trial learning rule used by the animal, confronting three challenges highlighted in Related Work: suboptimal or biased updates, the need to integrate external stimuli and multi-trial history, and the limits of fixed rule forms or static policy features. We demonstrate that our framework recovers ground--ruth rules in simulation, extends non-Markovian dynamics via recurrent architectures, improves held-out prediction accuracy on real behavioral data, and yields analy insights into animal learning. Together, these results support a data-driven framework for uncovering learning processes.
To knowledge, this is the first study to infer the learning rule function in de novo learning settings without assuming a specific model class. While neural network-based approaches have been proposed in non-de novo settings such as bandit tasks (21-25), structured sensory inputs and the need to learn new input-output mappings introduce additional challenges. Prior work has also modeled behavior by having RNNs directly output cognitive states or choices, but such models often act as black boxes that are difficult to interpret [21 (2)). Although recent studies have sought to balance ficxibility and interpretability by using RNNs represent classical model parameters [52]. we find that directly modeling parameter updates Aw leads to better ground truth reconstruction (Supp. Fig. 14). This advantage likely stems from the implicit regularization of neural networks, which biases learning dynamics toward smoother functions first in the feature learning regime [72184). That said, it remains unclear whether the inductive bias of our setup generalizes across various learning systems an open question we highlight as a future direction (see below). One promising result supporting our approach is that, a sense, implicitly regularizing weight updates aligns with the philosophy behind PsyTrack, which also acts at the level of weight changes (28).
Broader outlook for neuroscience and AI. By inferring learning rules directly from behavior without assuming fixed model structure, our approach offers a data-driven framework for understanding how animals learn de novo. This has several implications. First, interpreting the inferred learning rules could offer insights into learning behavior (e.g., suboptimal or history-dependent patterns), which may point to additional variables- such as trial history, internal states, or reward timing that are worth tracking in future behavioral learning studies. In turn, more accurate behavioral learning models could inform applications in health and conservation e.g. anticipating how animals respond to environmental changes or interventions [4 This also lays a foundation for more challenging settings with richer task structures and scaling to more expressive architectures. Second, de novo learning is ubiquitous in experimental neuroscience, where animals are often trained from scratch on novel tasks, yet remains understudied. By revealing structure in how animals learn during this phase, our method can inform experimental design and training protocols, supporting more efficient task training 19 20).
From an Al perspective, our work contributes to the growing interest in biologically- alligned learning systems 1 14. By uncovering suboptimality and history dependence in animal learning, we reveal deviations from standard artificial agents that could inform the design of more human- or animal-aligned AI. Relatedly, our approach may support the development of behavioral digital twins-Al models that mimic individual learning processes for simulation, personalization, or real- time prediction [85]. Finally, by leveraging the implicit regularization of neural networks to infer learning rules, our work raises questions about how such inductive biases interact with neural network-based system identification (see below).
Limitations and future directions. Pooling across animals. Neural network-based approaches are often data hungry, and reconstruction accuracy improves with more data (Fig. (Cig.gC). To address this, we pool across animals rather than fitting individual subjects. While pooling is common in neuroscience [86-88] it can obscure individual variability and inherit known limitations [89] Nonetheless, learning rule inference from behavior alone is underconstrained in low-data regimes, and pooling offers a practical mitigation strategy. This population-level approach aligns with broader trends in neuroscience and AI toward large-scale behavioral datasets.

---

Static vs. dynamic learning rules. Our current model assumes static weight update function. While RNNs may implicitly capture temporal changes-perhaps explaining why fitting separate RNNGLMs to the first and second halves of training did not yield improvement-future work could explicitly model dynamic learning rules to better disentangle non-Markovian dependencies from nonstationarity. An important direction is to investigate whether the observed suboptimal negative baseline arises from specific cognitive mechanisms - such as choice perseverance or forgetting [90] or from model mismatch. More broadly, suboptimal learning is common in the literature 92). especially in pathological populations [93 94], where applying our approach could be interesting.
Stochasticity, Our current learning model is deterministic. Capturing stochasticity in learning would be a valuable extension.
Generalization across datasets, tasks, and architectures We evaluated our method using the same preprocessed IBL. dataset [19]. This focus on relatively simple task was intentional: it mirrors widely used de novo learning paradigms in laboratory neuroscience, where animals are introduced to novel tasks with minimal prior structure. Such simplified tasks capture key hallmarks of biolog- ical learning-histor dependence, biases, and suboptimal strategies- whhle providing a tractable platform for reverse-engineering learning dynamies. In this sense, our work should be viewed as a stepping stone toward broader generalization: establishing validity in a controlled setting before extending to richer tasks and datasets. Indeed, demonstrate potential scalability, we tested higher- dimensional inputs and found that our approach continues to recover core qualitative features of learning rules, with performance improving as data availability increases (Supp. Table O These results indicate that our framework can extend beyond the simple tasks considered here.
Beyond extending to diverse tasks and datasets, another future avenue would be investigate how different architectures-and their associated inductive biases-affect inference accuracy, particularly under data limitations. Our framework currently fixes the decision model (c.g., standard models such as GLMs), which aids interpretability and follows common neuroscience practice but limits expressivity. In fact, for multiplicative decision processes, this assumption leads to a significant performance degradation (Supp. Table We view this modular approach- using a standard decision model while focusing on trial-by-trial weight updates- as a principled first step, consistent with strategies in neuroscience and machine learning where components are studied separately before joint inference. Extending the framework to jointly infer both the decision model and the learning rule an important direction for future work.
Overall, we introduce flexible framework for inferring learning rules directly from behavior, without assuming predefined form. By capturing non-Mat non-Markovian dynamics and uncovering insights into de novo learning, hope this approach opens new avenues for studying how animals learn in rich, structured environments.
Acknowledgement
The authors thank the International Brain Laboratory for data availability, Orren Karniol-Tambon for feedback on the manuscript, and Alex Riordan for helpful discussions. YHL was partially supported by the Fonds de recherche du Québec Nature et technologies (FRQNT). VG was supported the Porter Ogden Jacobus Fellowship at Princeton University, and by doctoral scholarships from the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Fonds de recherche du Québec Nature et technologies (FRQNT). JWP was supported by grants from the Simons Collaboration on the Global Brain (SCGB AWD543027), the NIH BRAIN initiative (9R01DA056404-04) an NIH R01 (NIH IR01EY033064), and a U19 NIH-NINDS BRAIN Initiative Award (U19NS
References
[1] Ivan Pavlov. Conditioned Reflexes. Oxford University Press, 1927.
[2] Robert A. Rescorla and Allan R. Wagner. A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcementt Classical Conditioning II: Current Research and Theory, page 64-99, 1972.
[3] Edoardo Fazzari, Donato Romano, Fabrizio Falchi, and Cesare Stefanini. Animal behavior analysis methods using deep learning: A survey. arXiv preprint 240.102005..4002, 2024.
10

---

[4] Sudarsini Tekkam Gnanasekar, Svetlana Yanushkevich, Nynke J Van den Hoogen, and Tuan Trang. Rodent tracking and abnormal behavior classification in live video using deep neural networks. In 2022 IEEE Symposium Series on Computational Intelligence (SSC7), pages 830-837. IEEE, 2022.
[5] Julie K Shaw and Sarah Lahrman. The human complexities. Canine and feline behavior for veterinary technicians and nurses, pages 88-105, 2023.
human-animal animal bond-a brief look at its richness and
[6] Gianluca Manduca, Valeria Zeni, Sara Moccia, Beatrice A Milano, Angelo Canale, Giovanni Benelli, Cesare Stefanini, and Donato Romano. Learning algorithms estimate pose and detect motor anomalies in flies exposed minimal doses of a toxicant. Iscience, 26(12), 2023.
[7] Benjamin L Hart. Behavioural defences in animals against pathogens and parasites: parallels with the pillars of medicine in humans. Philosophical Transactions of the Royal Society B: Biological Sciences, 336(1553) 2011.
[8] Dhanushi A Wijeyakulasuriya, Elizabeth W Eisenhauer, Benjamin A Shaby, and Ephraim M Hanks. Machine learning for modeling animal movement. PloS one, 15(7):e0235750, 2020.
[9] Jin Hou, Yuxin He, Hongbo Yang. Thomas Connor, Jie Gao, Yujun Wang, Yichao Zeng, Jindong Zhang, Jinyan Huang, Bochuan Zheng, et al. Identification of animal individuals using deep learning: A case study of giant panda. Biological Conservation, 242:108414 14, 2020.
[10] Ellen M Ditria, Sebastian and Rod M Connolly. Automating the analysis of fish abundance using object detection: optimizing animal ecology with deep learning. Frontiers in Marine Science, 7:429, 2020.
Michael Sievers, Eric L Jinks, Christopher J Brown,
[11] Rudresh Pillai, Rupesh Gupta, Neha Sharma, and Rajesh Kumar Bansal. A deep learning approach for detection and classification of ten species of monkeys. In 2023 International Conference on Smart Systems for applications in Electrical Sciences (ICSSES), pages 1-6.
IEEE, 2023.
[12] Simon RO Nilsson, Nastacia L. Goodwin, Jia Jie Choong, Sophia Hwang, Hayden R Wright, Zane C Norville, Xiaoyu Tong, Dayu Lin, Brandon Bentzley, Neir Eshel, et al. Simple behavioral analysis (simba)-an open source toolkit for computer classification of complex social behaviors in experimental animals. BioRxiv, pages 2020-04, 2020.
[13] Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C Love, Erin Grant, Iris Groen, Jascha Achterberg, et al. Getting aligned on represen- tational alignment. arXiv preprint arXiv:2310. 13018. 2023.
[14] Anthony Zador, Sean Escola, Blake Richards, Bence Ölveczky, Yoshua Bengio, Kwabena Boahen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, et al. Catalyzing next-generation artificial intelligence through neuroai. Nature communications, 14(1):1597, 2023.
[15] Peter Dayan and Nathaniel Daw. Decision theory, reinforcement learning, and the brain. Cognitive, Affective, & Behavioral Neuroscience, 8(4):429 453, 2008.
[16] Yael Niv. Reinforcement learning in the Journal of Mathematical Psychology, 53(3):139-
154, 2009.
[17] Richard S Sutton and Andrew G Barto. Fime-derivative models of pavlovian reinforcement.
Learning and computational neuroscience: Foundations of adaptive networks, 1988.
[18] Armin Lak, Emily Hueske, Junya Hirokawa, Paul Masset, Torben Ott, Anne E Urai, Tobias H Donner, Matteo Carandini, Susumu Tonegawa, Naoshige Uchida, al. Reinforcement biases subsequent perceptual decisions when confidence is low, widespread behavioral phenomenon. ELife, 9re49834, 2020.
[19] Zoe Ashwood, Nicholas A Roy, Ji Hyun Bak, and Jonathan W Pillow. Inferring learning rules from animal decision-making. Advances in Neural Information Processing Systems, 33:3442-3453, 2020.
[20] Victor Geadah and Jonathan W Pillow. Inferring learning rules during de novo task learning. bioRxiv, pages 2025-09, 2025.
[21] Li Ji-An, Marcus Benna, and Marcelo G Mattar. Automatic discovery of cognitive strategies with tiny recurrent neural networks. bioRxiv, pages 2023-04, 2023.

---

[22] Kevin Miller, Maria Eckstein, Matt Botvinick, and Zeb Kurth-Nelson. Cognitive model dis- covery via disentangled mns. Advances in Neural Information Processing Systems, 36:61377- 61394, 2023.
[23] Maria K Eckstein, Christopher Summerfield, Nathaniel D Daw, and Kevin Miller. Predictive and interpretable: Combining artificial neural networks and classic cognitive models to understand human learning and decision making. BioRxiv, pages 2023-05, 2023,
[24] Amir Dezfouli, Hassan Ashtiani, Omar Ghattas, Richard Nock, Peter Dayan, and Cheng Soon Ong. Disentangled behavioural representations. Advances in neural information processing systems, 32, 2019.
[25] Pablo Samuel Castro, Nenad Tomasev, Ankit Anand, Navodita Sharma, Rishika Mohanta, Aparna Dev, Kuba Perlin, Siddhant Jain, Kyle Levin, Noémi Éltetó, et al. Discovering symbolic cognitive models from human and animal behavior. bioRxiv, pages 2025-02, 2025.
[26] Joanna C Chang, Matthew G Perich, Lee Miller, Juan Gallego, and Claudia Clopath. De novo motor learning creates structure in neural activity space that shapes adaptation. bioRxiv, 2023.
[27] Davin Greenwell, Samia Vanderkolff, and Jacob Feigh. Understanding de novo learning for brain-machine interfaces. Journal of Newrophysiology, 129(4):74--750, 2023.
[28] Nicholas A Roy, Ji Hyun Bak, Athena Akrami, Carlos Brody, and Jonathan W Pillow. Effi- cient inference for time varying behavior during learning. Advances in neural information processing systems, 31, 2018.
[29] Sebastian A. Bruijns, Kcénia Bougrova, Inês C. Laranjeira, Petrina Y. P. Lau, Guido T. Meijer, Nathaniel J. Miska, Jean-Paul Noel, Alejandro Pan- Vazquzz Noam Roth, Karolina Z. Socha, Anne E. Urai, and Peter Dayan. Dissecting the complexities of learning with infinite hidden markov models. bioRxiv 2023.12.22.573322
December 2023.
[30] Nicholas A Roy, Ji Hyun Bak, Athena Akrami, Carlos D Brody, and Jonathan W Pillow. Extracting the dynamics of behavior in sensory decision-making experiments. Neuron, 109(4):597-610, 2021.
[31] International Brain Laboratory. A standardized and reproducible method to measure decision- making in mice. BioRxiv, page 909838, 2020.
[32] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using mn encoder- decoder for statistical machine translation. arXiv preprint arXiv:1406. 1078, 2014.
[33] Ronald J Williams. Simple statistical gradient- algorithms for connectionist rein- forcement learning. Machine learning, 8:229-256, 1992.
[34] Samuel Liebana Garcia, Aeron Laffere. Chiara Toschi, Louisa Schilling, Jacek Podlaski, Matthias Fritsche, Peter Zatka-Haas, Yulong Li, Rafal Bogacz, Andrew Saxe, al. Striatal dopamine reflects individual long-term learning trajectories. BioRxiv, pages 2023-12, 2023.
[35] Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al.
A deep learning framework for neuroscience. Nature neuroscience, 22 1761177177 2019.
[36] Guangyu Robert Yang and Xiao-Jing Wang. Artificial neural networks for neuroscientists: a primer. Neuron, 1008 1070, 2020.
[37] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James DiCarlo. cerfornance-optimizze hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23):8619886)) 8624, 2014.
[38] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap. Pi- otr Mirowski, Alexander Pritzel, Martin J Chadwick, Thomas Degris, Joseph Modayil, et al. Vector-based navigation using grid-like representations in artificial agents. Nature, 557(7705):429 2018.
[39] Christopher J Cueva and Xue-Xin Wei. Emergence of grid representations by training recurrent neural networks to perform spatial localization, arXiv preprint arXiv: urXiv:1803.000 1803.07770, 2018.
12

---

[40] James CR Whittington, Timothy H. Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy EJ Behrens. The tolman-eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation, Cell, 183(5):129-- 1263, 2020.
[41] Danil Tyulmankov, Guangyu Robert Yang, and LF Abbott. Meta-learning synaptic plasticity and memory addressing for continual familiarity detection. Neuron, 110(3):44-557, 2022.
[42] Valerio Mante, David Sussillo, Krishna V Shenoy, and William Newsome. Context- dependent computation by recurrent dynamics in prefrontal cortex. nature, 503(7474):78-84) 2013.
[43] Marino Pagan, Vincent D Tang, Mikio C Aoi, Jonathan W Pillow, Valerio Mante, David Sussillo, and Carlos D) Brody. Individual variability of neural computations underlying flexible decisions. Nature, 639(6054):421-422) 2025.
[44] Jane X Wang, Zeb Kurth-Nelson, Dharshan Kumaran, Dhruva Tirumala, Hubert Soyer, Joel Z. Leibo, Demis Hassabis, and Matthew Botvinick. Prefrontal cortex as a meta-reinforcement learning system. Nature neuroscience, 21(6):860-868, 2018.
[45] Valeria Fascianclli, Fabio Stefanini, Satoshi Tsujimoto, Aldo Genovesio, and Stefano Fusi. Neural representational geometry correlates with behavioral differences between monkeys. bioRxiv,, pages 2022-10, 2022.
[46] Kristopher T Jensen, Guillaume Hennequin, and Marcelo G Mattar. A recurrent network model of planning explains hippocampal replay and human behavior. Nature neuroscience, 27(7):1340-1345 2024.
[47] Joshua C Peterson, David D Bourgin, Mayank Agrawal, Daniel Reichman, and Thomas L Griffiths. Using large-scale experiments and machine learning discover theories of human decision-making Science, (7206547):1209-1222 2021.
[48] Mingyu Song, Yael Niv, and Mingbo Cai. Using recurrent neural networks to understand human reward learning. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 43, 2021.
[49] Paul Jaffe, Russell A Poldrack, Robert Schafer, and Patrick G Bissett. Modelling human behaviour in cognitive tasks with latent dynamical systems. Nature Human Behaviour, 7(6)998611000 2023.
[50] Ionatan Kuperwajs, Heiko H Schütt, and Wei Ji Ma. Using deep neural networks as guide for modeling human planning. Scientific reports, 13(1):20269, 2023.
[51] Milena Rmus, Ti-Fen Pan, Liyu Xia, and Anne GE Collins. Artificial neural networks for model identification and parameter estimation in computational cognitive models. PLOS Computational Biology, 20(5)=e10221) 2024.
[52] Yoav Ger, Eliya Nachmani, Lior Wolf, and Nitzan Shahar. Harnessing the flexibility of neural networks to predict dynamic theoretical parameters underlying human choice behavior. PLoS Computational Biology, 20(1 1001)=e101 1678, 2024.
[53] Jeffrey C. Magee and Christine Grienberger. Synaptic Plasticity Forms and Functions. Annual Review of Neuroscience, 43(():95-117,, jul 2020.
[54] Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, and Johanni Brea. Eligi- bility Traces and Plasticity Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules. Frontiers in Neural Circuits, 12:53, jul 2018.
[55] Anne E Urai, Anke Braun, and Tobias H Donner. Pupil-linked arousal driven by decision uncertainty and alters serial choice bias. Nature communications, 8(1):14637, 2017.
[56] Ari S Morcos and Christopher D Harvey. History-dependent variability in population dynamics during evidence accumulation in cortex. Nature neuroscience, 19(12):1622-1681 2016.
[57] Athena Akrami, Charles D Kopec, Mathew E Diamond, and Carlos D Brody. Posterior parictal cortex represents sensory history and mediates its effects on behaviour. Nature, 554(7692):66-377, 2018.
[58] Sukbin Lim, Jillian I. McKee, Luke Woloszyn, Yali Amit, David J Freedman, David L. Sheinberg, and Nicolas Brunel. Inferring learning rules from distributions of firing rates in cortical neurons. Nature neuroscience, 1804))1804-1110, 2015.
13

---

[59] Jacob Portes, Christian Schmid, and James M Murray. Distinguishing learning rules with brain machine interfaces. Advances in neural information processing systems. 35:25937-259501 2022.
[60] Daniel R Kepple, Rainer Engelken, and Kanaka Rajan. Curriculum learning as a tool to uncover learning principles in the brain. In International Conference Learning Representations. 2021.
[61] Aran Nayebi, Sanjana Srivastava, Surya Ganguli, and Daniel L Yamins. Identifying learning rules from neural network observables. Advances in Neural Information Processing Systems, 33:2639-2650, 2020.
[62] Yash Mehta, Danil Tyulmankov, Adithya Rajagopalan, Glenn Turner, James Fitzgerald, and Jan Funke. Model based inference of synaptic plasticity rules. Advances in Neural Information Processing Systems, 37:48519-445 19-44400 2024.
[63] Basile Confavreux, Friedemann Zenke, Everton Agnes, Timothy Lillicrap, and Tim Vogels. A meta-learning approach to (re) discover plasticity rules that carve a desired function into neural network. Advunces in Neural Information Processing Systems, 33: 33:16398-16408 2020.
[64] Basile Confavreux, Poornima Ramesh, Pedro J Goncalves, Jakob H Macke, and Tim Vogels. Meta-learning families of plasticity rules in recurrent spiking networks using simulation-basee inference. Advances in Neural Information Processing Systems, 36: 16:1355 3555-135588, 2023.
[65] Poornima Ramesh, Basile Confavreux, Pedro J Goncalves, Tim P Vogels, and Jakob H Macke. Indistinguishable network dynamics can emerge from unalike plasticity rules. bioRxiv, pages 2023-11, 2023.
[66] David Bell, Alison Duffy, and Adrienne Fairhall. Discovering plasticity rules that organize and maintain neural circuits. Advances in Neural Information Processing Systems, 37:40732-
40751, 2024.
[67] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. Advances in neural information processing systems, 29, 2016.
[68] Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel. RI2: Fast reinforcement learning via slow reinforcement learning. 2016. URL http://arxiv. org/ubs/161 1. .007779, 2016.
[69] Thomas Miconi, Kenneth Stanley, and Jeff Clune. Differentiable plasticity: training plastic neural networks with backpropagation. In International Conference on Machine Learning. pages 3559-35663 PMLR, 2018.
[70] Navid Shervani- Tabar and Robert Rosenbaum. Meta-learning biologically plausible plasticity rules with random feedback pathways. Nature Communications, 14(1): 2023.
[71] David B Kastner, Eric A Miller, Zhuonan Yang, Demetris K Roumis, Daniel F Liu, Loren M Frank, and Peter Dayan. Spatial preferences account for inter- animal variability during the continual learning of dynamic cognitive task. Cell reports, 39(3), 2022.
[72] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
[73] Lenaic Chizat, Edouard Oyallon, and Francis Bach. On lazy training differentiable program- ming. Advances in neural information processing systems, 32, 2019.
[74] Timo Flesch, Keno Juechems, Tsvetomira Dumbalska, Andrew Saxe, and Christopher Sum- merfield. Rich and lazy learning of task representations in brains and neural networks. BioRxiv, pages 2021-04, 2021.
[75] Thomas George, Guillaume Lajoie, and Aristide Baratin. Lazy hasty: linearization in deep networks impacts learning schedule based on example difficulty. TMLR, 2022.
[76] Lukas Braun, Clémentine Dominé, James Fitzgerald, and Andrew Saxe. Exact learning dynamics of deep linear networks with prior knowledge. Advances in Neural Information Processing Systems, 35:6615-6629, 2022.
[77] Mohammad Pezeshki, Oumar Kaba, Yoshua Bengio, Aaron C Courville, Doina Precup, and Guillaume I ajooe.. Gradient starvation: A learning proclivity neural networks. Advances in Neural Information Processing Systems, 34, 2021.
14

---

[78] Aristide Baratin, Thomas George, César Laurent, R Devon Hjelm, Guillaume Lajoie, Pascal Vincent, and Simon Lacoste-Julin Implicit regularization via neural feature alignment. In International Conference on Artificial Intelligence and Statistics, pages 2269-2277. PMLR, 2021.
[79] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International conference on machine learning, pages 5301-5310. PMLR, 2019.
[80] Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu. Towards understand- ing the spectral bias of deep learning. arXiv preprint 1rxiv:191,.01198, 2019.
[81]
Alexander Atanasov. Blake Bordelon, and Cengiz Pehlevan. Neural networks kernel learners: The silent alignment effect. arXiv preprint arXiv:2111.00054, 2021.
[82] Abdulkadir Canatar, Blake Bordelon, and Cengiz Pehlevan. Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks. Nature communicationss 2211)29114 2021.
[83] Melikasadat Emami, Mojtaba aahraee-Ardakan, Parthe Pandit, Sundeep Rangan, and Alyson K Fletcher. Implicit bias of linear mns. In International Conference on Machine Learning, pages 2982-2992. PMLR, 2021.
[84] Andrew Gordon Wilson. Deep learning is not so mysterious or different. arXiv preprint arXiv:2503.0211 2025.
[85] Evangelia Katsoulakis, Qi Wang. Huanmei Wu, Leili Shahriyari, Richard Fletcher, Jinwei Liu, Luke Achenie, Hongfang Liu, Pamela Jackson, Ying Xiao, et al. Digital twins for health: a scoping review. NPJ digital medicine, 7(1):77, 2024.
[86] Zoe Ashwood, Aditi Jha, and Jonathan W Pillow. Dynamic inverse reinforcement learning for characterizing animal behavior. Advances in neural information processing systems, 35:29663-29678 2022.
[87] Zoe C Ashwood, Nicholas A Roy, Iris R Stone, International Brain Laboratory, Anne E Urai, Anne K Churchland, Alexandre Pouget, and Jonathan W Pillow. Mice alternate between discrete strategies during perceptual decision-making. Nature Neuroscience, 2522):001-21,, 2022.
[88] Scott S Bolkan, Iris R Stone, Lucas Pinto, Zoe C Ashwood, Jorge M Iravedra Garcia, Alison L Herman, Priyanka Singh, Akhil Bandi, Julia Cox, Christopher A Zimmerman, et al. Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state. Nature neuroscience, 25(3):345-357, 2022.
[89] Sharlen Moore and Kishore V Kuchibhotla. Slow or sudden: Re-interpreting the learning curve for modern systems neuroscience. IBRO Neuroscience Reports, 13:9-14, 2022.
[90] Ronald L Davis and Yi Zhong. The biology of forgetting perspectivve. Neuron, 95(3):490- 503, 2017.
[91] Manuel Molano-Mazón, Yuxiu Shao, Daniel Duque, Guangyu Robert Yang, Srdjan Ostojic, and Jaime De La Rocha. Recurrent networks endowed with structural priors explain suboptimal animal behavior. Current Biology, 33(4):622-6)) 638, 2023.
[92] Tzuhsuan Ma and Ann M Hermundstad. A vast space of compact strategies for effective decisions. Science Advances, 10(25):eadj4064, 2024.
[93] Amir Dezfouli, Kristi Griffitss, Fabio Ramos, Peter Dayan, and Bernard W Balleine. Models that learn how humans learn: The case of decision- maaiinnn--annnggaa and its disorders. PLoS computa- tional biology, 15(6)=e100600: 2019.
[94] M Ganesh Kumar, Adam Manoogian, Billy Qian, Cengiz Pehlevan, and Shawn A Rhoads. Neurocompuuaaionall underpinnings of suboptimal beliefs in recurrent neural network-based agents. bioRxiv, pages 2025-03, 2025.
[95] George Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4):303-314, 1989.
[96] Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural networks, 4(2):251-257, 1991.
15

---

[97] Allan Pinkus. Approximation theory of the mlp model in neural networks. Acta numerica, 8:143-195, 1999.
[98] MH Hassoun. Fundamentals of Artificial Neural Networks. The MIT Press, 1995.
[99]
Simon Haykin. Neural networks: comprehensive foundation. Prentice Hall PTR, 1998.
[100] Zhou Lu, Hongming Pu, Feicheng Wang. Zhiqiang Hu, and Liwei Wang. The expressive power of neural networks: view from the width. Advances in neural information processing systems, 30, 2017.
[101] Ken-ichi Funahashi and Yuichi Nakamura. Approximation of dynamical systems by continuous time recurrent neural networks. Neural networks, 6(6):801-806, 1993.
[102] Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
[103] Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts,
2018.
[104] Mario Geiger, Stefano Spigler, Arthur Jacot, and Matthieu Wyart. Disentangling feature and lazy training in deep neural networks. Journal of Statistical Mechanics: Theory and Experiment, 2020(1 )): 13301, 2020.
[105] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pegh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[106] Pauli Virtanen, Ralf Gommers, Travis Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):261-272, 2020.
16

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: To make this easier for the readers, have referred to the pertinent figures and sections under "Main contributions" in Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Details on limitations and future work are discussed in the 'Limitations and future works' subsection in Discussion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [Yes]
Justification:
We provide detailed assumption and proof for Proposition | in Supp.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
Training details are provided in Supp. Moreover, code is available attps://githhn com Heee/Heeeeeeaaa Y-han- Lin/ Inf affrLearningANNG..
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
18

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We use IBI. datasets, which are publicly available. Our code is available https: bttpss//giihubb www/ Yataaa--uaaaauuuuaaa http:///wwwnneeeeaaaaaaaallAACC
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
aublic/guudees/Coeettbii.ss
www.ssssinoooiiiioiooooo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. wwtp:/cccpblccggguiieeeeee Stw.eeee///ooSeeesssSeiiee..
www.sssssssiiggoooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details are provided Supp.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
We have included this information in all applicable figures and tables. In particular, p-values for log- likelloodoo comparisons are provided in Table with details of the t-test and pross-validatee procedure described in Supp. C
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Information pertaining to computing resources and simulation time can be found in Supp. C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't it make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httppp/:/eaurrrss.ccpppppal/aaad
Answer: [Yes]
Justification:
We have carefully read the NeurlPS Code of Ethics and attest that the research
conforms.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This research advances our understanding of animal learning mechanisms and contributes to the development interpretable models for behavior. We do not anticipate any immediate ethical or societal impacts. However, over time, these findings could influence related fields such as neuroscience and machine learning, which may indirectly affect society depending on how these technologies are applied.
20

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No specific safeguards were implemented, since this work is focused on basic research better understand animal learning, as explained above. The model is not intended for deployment or real-world decision decision-making, As such, no foresecable misuse or ethical risks require mitigation at this stage.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Please see Supp.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
21

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thcode com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
22

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were used solely for writing, editing, and formatting assistance in this work. We also briefly used Deep Research to assess the coverage of our related work discussion and found that our initial literature review was already comprehensive.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https://neur)) http:///...c.o.oeeeeeeee/////000 for what should or should be described.
23

---

A Theoretical justifications
To gain theoretical insights into why this approach could work, we consider two key questions: (1) Do neural networks have the capacity to approximate the learning rule, i.e. the weight update function? (2) Since weight updates are not directly observable, can they be identified from behavioral data? Question (1), assuming well-behaved weight update functions, is addressed by universal approximation theorems [95 [01]. For question (2), we start by considering very simple idealized setting in the proposition below and leave comprehensive theoretical investigations into identifiability for future work.
Proposition 1. For simplicity, let w. E for t E {0,1}, and assume ~ Bernoulli(Py.. where
Py. = Plyy = W., 3T] oow,er) with o(2) I+e denoting the sigmoid function.
We further make the following simplifying assumptions: (1) Aw, Wt+1 We depends only on (We, Dt. 3t)) (2) For each inputs 2 E (-1, are drawn i.i.d. across samples from the uniform distribution over this set, i.e., Px,: = =1) P(z, 1) = 2.2 (3) The same initial weight wo is
used across N independent repetitions. (4) For all sample i, we assume p(() E [k, - for some arbitrary 0 <
(5) The prior on is smooth, strictly positive, and does not concentrate
the boundary of the parameter space. Then, as N approaches infinity, the standard error (SE) of Awo W1 Wh across samples approaches
Proof. The log- logeliill for a single observation at time is:
(w;) Ye log pg. + (1 ye log(1 P..)
By taking the second derivative, arrive at:
(l,, PPo p
Ow?
P1 Ps.);
where x² is dropped due to the assumption that x E {-1, 13. Summing over N independent observations, the Fisher information for w, is:
I(w.)
Σ
p p(t)) Me
yr
By the Bernstein-von Mises (BvM) theorem (see [102] ) the posterior distribution of w, given the data is asymptotically normal:
क. (W; data) 4 N We I(we)
where We is the MLE of Wt. Thus, by ByM theorem, in the limit N + co, the posterior variance is asymptotically equal to:
Var(w:) ΣN p(f (1 (1(())))
and applying assumption (4):
Var(w:) 5 Ne(1 c))
and
SE(w:) S Ne(1 6)
24

---

Since Awo W1 Wo, the variance of Awo can be expressed as:
Var(Awo) = Var(wn) + Var(wo) 2 Cov(w1, wo)
Using the Cauchy-Schward inequality, the covariance term is bounded:
Cov(w2,wo)) I v /Va((w)) Var(wa).
Thus, have:
Var(Awn) S Var(wi) + Var(wo) 2 Var(w)) Var(w)).
Substituting the variance bounds:
Var (ww)) S Ne(1 c) + Ne(1 c) + +2/ Ne(1 -e) E)
This simplifies to:
(ar() (Nee(Lo)++ VV Ne(11 -
1
Taking the square root to get the SE:
1 SE(Awo) S Nee11 - + V /NNel -c]
As N - oo, both terms approach 0, so SE(Awo) -
Despite the restrictive assumptions, this proposition demonstrates (I) identifiability in highly simple settings to motivate future theoretical work and (2) identifies factors that contribute to the Fisher information. These factors include the number of animals pooled, N (see Fig. and initial weights, Wo (see Supp. Table 3 We also note that the third assumption could be approximately achieved by pretraining animals a common performance level by pooling animals that exhibit similar initial behavior.
25

---

B Learning rules considered for simulated data
In simulated data, we model weight updates as follows:
+++ + Aw.,
(9)
where Aw, is determined by a specific ground truth learning rule. We remind readers that variables were defined in Section 2
For Markovian learning rules, consider the classical REINFORCE 33 B.I:
(more details in Supp.
Aw, r(y,, V logp(u: Xt, We) = =r(,, 2r (1 Pvs Jrer c== = +1, G/=L = -1,
(10)
where Py. : p(ye X,, W+).
Results for simulated data using the classical REINFORCE rule were shown Fig. 2 key property of this rule-captured well by our method is the clear difference in weight updates following correct versus incorreet choices. To further test our approach, we considered an alternative rule that lacks this characteristic: maximum likelihood rule (Supp. Fig. (). This can be viewed analogous to supervised learning, where the correct label is known and the weight update aims to increase the probability of choosing the correct action:
Aw; x Ww log logp(zi
Xt: w)
=z,
Pz, )ai,
6 +1, 6x -1,
(1I)
which doesn't depend on the action and reward.
For non-Markovian learning rule in Fig. we consider a modified REINFORCE with an "elibility- trace-like" factor (see Supp. B.2
Aw O ((),, 21) Σ Cr Pyr-a )
(12)
B.1 REINFORCE
We present the classical REINFORCE 33
Aw, re(y1, Vw, w.),
(13)
which is derived as follows.
We want to maximize the expected reward (expectation across the agent's stochastic behavior) by computing its gradient:
ww, E P wry =)]==w. Vw. / rry,, P(ucl w;)dy,
21) Plye xe, weddy:
(b)
r(y., 24 P(y w. V log (ag, we)dy:
E P. (ww V V., log P (22,, w.)).
(14)
where (a) is because r(ye, =x) is independent of w, once is given, and (b) is from the log-derivative
trick. We can evaluate the right-hand-side mand-side with Monte Carlo integral by assuming the animal's choice y is sampled according its policy ( (T,, we). which results in Eq. 13
26

---

B.2 REINFORCE with "eligibility trace"
Suppose we change the task setting and the reward ry now depends up to past data points, i.e.
= -
S:,, St)) then:
w. Ep(x- si folll wa (yt- st)]
Vw. /
2- Sit P(y P --
rUt 21
St V Wo St
14 4 v Po
W.-a],
(15)
where (a) is because
is independent of once V1-94 is given, and (b) is from the log-derivative trick: (c) we assume Wi-e 2 w, and also P(ye-s:lzz-ss) Wt-S:)
I1 P Wi-s) from the task setup. Again, we apply Monte-Care approximation:
S
Aw, X r
Vw.., Po Wt-s).
(16)
For Fig. we used S 10 and defined r. rr (ytt-::tt:: S:t) as if at least half of the recent trials are correct and ( otherwise; we found similar results for other choices too. We refer to it as Chigibility-trace-lkee although it may not arise from eligibility traces, since the reward is combined with input/outputtdepeeddett factor that accumulates over multiple time steps.
27

---

C Additional details on model and data
Simulated data. We simulate a pool of animals, each following the same underlying learning rule. In Supp. Fig. I1 we also explore populations with mixed weight update functions, where pooling tends to recover an in-between weight update function. Even when animals follow the same rule, their learning trajectories differ due to the stochasticity of received inputs and behavior.
For each simulated animal, the learning process unfolds over trials = 0, I....,TT stimulus 8, is drawn uniformly from discrete values in [-2. 2] with 0.25 increments, excluding O (except for Fig. 2 which includes 0); varying the increment size did not affect our conclusions. The correct label or answer is defined as = [s, > 0j. Decision weights are updated according to W(+1 W + Awe, where Aw; is computed from the ground fround-truth learning rule being tested (see
T' as follows. Each
Supp. B). The agent's 's binary choice ys E {0, 1) is sampled from a Bernoulli GLM parameterized
by Wt, as described in Section 2 A reward is given if the choice matches the correct answer:
= [yx = 2z).
For the total number of trials T, use T 500 by default (to speed up simulations), but also demonstrate that our conclusions hold for longer T (T' = 8000) in Supp. Fig. [0] We set the learning rate a such that the animal's performance reaches a saturation regime- - typically when the stimulus weight approaches a value of 3 for this task. The GLM includes a stimulus weight that is updated over trials, and fixed bias weight sampled randomly the start of each simulation from {-1,0, 13. We simulate a pool of 10 to 1000 animals to observe the influence of data availability on reconstruction accuracy in Fig. 2 We evaluate model performance using 5-fold rrossvvaidatiin, where each fold holds out a subset of animals entirely. This ensures generalization across individuals, consistent with the underlying assumption of pooling - that animals share commonalities which can be learned from a subset and applied to held-out animals.
IBL data. We use the publicly available International Brain Laboratory (IBL) dataset 31, and follow the same preprocessing pipeline as [19], with details available in their accompanying code. For dataset details and learning curves, refer to Supp. and Fig. S1 in 19]. In other words, we used the same preprocessed dataset as [19] (N = 12). for benchmarking purposes, For evaluation, we adopt two cross-validation strategies, and both led to similar trends (Tables I and 2). First, under the pooling assumption that a shared learning model generalizes across animals, we use K-fold -ross-validation with animal-held-out splits We fit the model using the first 10000 trials per animal to capture the full learning trajectory, including the saturation phase. For each fold, we report the average validation log- likkliieoood across four random seeds and conduct paired f-tests across matched animals to obtain the p-values reported in Table | The paired design accounts for variability in individual animals' performance: animals who learn well are easier to predict, while animals who have not learned the task are harder to model (see Fig. S1 in (19). Second, to assess temporal generalization, we adopt future-data holdout strategy [003. We train up to the point where the learning curve inflects (around trial 5500), ensuring that the evaluation data reflect ongoing learning rather than the saturation regime. This also avoids cutting off training too early, where little learning signals are present. We then evaluate performance on the next 500 trials beyond that point, although similar trends were observed when using 200 or 1000 trials instead. As above, we compute paired t-tests on
log- likelihosoo to assess significance.
Learning model, initial GLM weights, and hyperparameters. Our learning rule model is depicted in Fig. DC The network takes input the current trial stimulus 8,, choice 1/,, reward rt, and the current weight w,, For the IBL data, we additionally include binary inputs indicating daybreak and stimulus side; the trends in Table remain unchanged when these inputs are omitted. The network then outputs inferred weight update. For choice and reward, we input the animal's actual choice and reward rather than those generated by the model, since these are the signals the animal would have access to when updating its weights; this approach is also consistent with previous studies. The DNN architecture consists of two hidden layers with 32 units each, while the RNN uses a single recurrent layer with 32 hidden units. All architectural components and training settings are treated as hyperparameters and tuned via cross-validation by animal held-out. All neural network weights and biases were treated as trainable parameters.
To estimate the initial decision weights Wo. we treat them as trainable parameters but initialize them using a psychometric curve fit to the first 100 trials. Specifically, we compute the empirical probability of choosing y as function of stimulus value, which defines a psychometric curve.
28

---

Since we model the choice probability o(w r), where is the logistic sigmoid, we apply the miverse-iigmoid (logit) transformation the empirical probabilities, yielding linear target b b= logit(p(y = 1)))). We then fit via least-squares regression to the equation w = b using the stimulus values. We investigate the sensitivity of performance to incorrect Wo Table [ As mentioned, to enable faster simulations and repeated robustness tests, we used shorter trial sequences. Because these provide fewer early samples, we initialized with a known initial weight. That said, applying our method to unknown initial weight still led to accurate recovery (Supp. Fig. [OP. For IBL data, however, we instead initialize = 0, following [19], since animals typically start at chance-level performance; we also verified that training wo did not significantly impact our results on IBL data.
We tune hyperparameters, including the number of hidden units, number of layers, learning rate, and number of training epochs. We used batch size of 1 for simulated data. For real IBL data, we trained on all animals jointly; using batch size of instead did not affect our conclusions. Cross- validation ensures generalization and helps mitigate overfitting, in addition to the fact that neural networks feature learning regime inherently favor smooth function approximators via implicit regularization (see Discussion). These implicit biases are themselves controlled by architectural choices and training regimes (e.g., depth, width, and training duration) [84 [04], which justifies the need for hyperparameter tuning. While do not apply any explicit regularization in this work, exploring the interplay between explicit and implicit regularization on learning rule identifiability remains an interesting direction for future research. Validation-driv model selection is crucial to avoid recovering an incorrect learning rule that merely overfits the training data achieving high training likeliieldoood but poor generalization to held-out data.
Computational details. Our code is available https://ght We implemented our models in Py Torch v1.13.1 [10S. Simulations were run on a machine equipped with a 12th Gen Intel(R) Core(TM) i7-12700H processor (14 cores, 20 threads) with a maximum clock speed of 2.30GHz. Training averages around 2 hours per run. We used the Adam optimizer with a default learning rate of le 3, and binary cross-entropy loss, which corresponds to the negative log- likelihood under Bernoulli model. For statistical analysis and significance testing, we used tools from the SciPy package [06]..
com cww/Hellll...ooo Hete/Heleeea-eee Yuhan Liu/
InferLearningANNGG.
29

---

D Additional results
Fig. repeats Fig. 2 for additional bias weight values. Fig. shows the fitted bias weight updates for the IBL dataset. Table echoes the trends in D with future data heldout instead of animal- level heldout. Additionally, we also demonstrate our main findings are robust across a variety of conditions: (i) learning rule without the decision outcome asymmetry (Supp. Fig. 6 (ii) recovery
of an in-between learning rule when mixing update functions across individuals (Supp. Fig. [T]; (iii) different initial stimulus weights (Supp. Fig. 9 (iv) longer trial lengths (Supp. Fig. T0 We also show sensitivity to the initial weight estimate (Supp. Table 3) and degradation of recovery under added noise and incorrect initial weight estimates (Supp. Table 4 5nd Comparisons for parameterizing the weights w versus the updates Áw are provided in Fig. Finally, we show the uncovered weight trajectories in Fig. 15 and 16
During initial rebuttal experiments, we observed slightly higher performance for the Transformer variant compared to the RNN under limited training epochs. In the final analysis with full training epochs, this trend no longer holds performance across architectures comparable (p 7.44e 1). We note that ENN-Transformer comparison is not central to our main contribution. We hypothesize that the comparable performance may be due to the small sample size (12 animals, as in (19)). We anticipate that the scalability advantages of Transformer and state-space models (e.g. Mamba) will become increasingly important as we extend to larger datasets.
Whias -1
Wbias
Wblas
Markovian learning rule:
coneest IEsFrett Corrett
Learning rule Max. Likellhood aw
Aw,
incorectt Correct
0.05
incorrett
fitte
x
fitted
fitted
fitted
0.13
Tria
Wm- nnsmmett
comnect
WHO
correct
W-O
mecmmstt
connect
incoment
0.00
Figure 6: Similar to Fig. 2 but repeated for Max. Likelihood as the ground truth in simulated data. Again, we see that the reconstruction successfully captures key characteristics: the slowing of learning at higher weights, the increase with stimulus amplitude, and the lack of clear separation between correct versus incorrect decisions.
30

---

A
Ground truth AW after correct choice
Bias leftward
Neutral
Bias rightward
Wbias = -1
Wbias 0
Wbias =
3
3
3
11
11
1
-2
-1
-2
-1
-2
-1
Stimulus
Stimulus
Stimulus
B
true
true
true
after correct y
after incorrect y
w=-1
AW
0.1
AW
AN
w=0
W=
Stimulus
Stimulus
Stimulus
fitted
fitted
fitted
AW
AWA
AW
0.0
-2
2
Stimulus
Stimulus
Stimulus
Figure 7: Similar Fig. but repeated while holding at other bias weight values simulated data. The trends in Fig. 2 hold, and our reconstruction also captures the side asymmetry when the bias weight is nonzero.
31

---

Wati REINFORCE (fitted)
Watm REINFORCE (fitted)
Wstim 2 REINFORCE (fitted)
AWblas
AWblas
AWblas
0.0
WAIES WDIBS
-0.7
Stimulus
Stimulus
Stimulus DNNGLM
DNNGLM
DNNGLM
0.7
AWbias
AWbias
AWbias
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
05
1.0
Stimulus
Stimulus
Stimulus
Figure 8: Bias weight update function closely resembles REINFORCE. Similar to Fig. 5 which visualizes the stimulus weight update function, we plot here the inferred update function for the bias weight. We plot REINFORCE (fitted on the DNNGLM) the top row and DNNGLM on the bottom row. The fitted neural network model captures key qualitative features of the REINFORCE update rule for the bias weight.
Wbias = -1 true
Wbias =
Wbias = true
true
AW
W
MV
0.0
Stimulus
Stimulus
Stimulus
fitted
fitted
fitted
0.1
NW
DW
Stimulus
Stimulus
Figure 9: Different initial stimulus weight Wo- (A) In our main results for simulated data, we initialized with stimulus weight over time, thereby covering a wide range of stimulus weights during training. Here, we instead randomly initialize ~ U[-2, 2] (so different animals begin with different initial weights) and observe similar overall trends in the learned weight update function. Plotting convention follows that of Fig. 2
2 so that the animal starts with poor performance and learns
32

---

Wbias = -1
Wbias true
Wbias 1
0.025
AW
AV
AV
0
2
Stimalus S
Stimulus S
Stimulus S
fitted
fitted
fitted
AW
0.012
NV
AW
Stimulus
Stimulus
Stimulus 5
Figure 10: Trends observed in Fig. hoold for longer trial sequences (T 8000). Plotting conventions are identical to those in Fig. for simulated data.
Comparison
Heldout LL
p-val
REINFORCE vs. DNNGLM
-2924.4 vs.
-2905.8
1.41e 2
DNNGLM RNNGLM
-2905.8
-2881.2
2.24e 3
REINFORCE vs. nonnegative base
2924.4 vs.
-3272.4
1.08e 9
REINFORCE (history) vs. RNNGLM
-2914.8 vs. -28112
5.61c 4
DNNGLM (history) vs. RNNGLM
-2800.2 vs.
-2881.2
3.52e
tinyRNN vs. RNNGLM
2893.8 vs.
288112
3.38e
Table 2: Future-data prediction log-lik iigliiklihoods (higher is better) for different learning models, along with p-values from paired t-tests matched across mice. These results mirror the trends observed in Table
Initial stimulus weight Wo
RMSE of reconstructed Aw
2
0,0130
0.0244
2
0,0287
Table 3: Impact of initial stimulus weight Wo on reconstruction accuracy. We measure reconstruc- tion RMSE of the inferred Aw when varying the ground truth initial weight wo in simulated data, keeping all other factors fixed. Initializing with Wi to saturation and poor identifiability. This pattern is consistent with Fisher Information analysis (Supp. A): having more datapoints around weight values near w correspond to choice probabilities near 0.5, which maximize Fisher Information and enhance identifiability of learning rules. Starting from Wp = -2 ensures the learning trajectory passes through this region, enabling more accurate recovery of the update function. This trend holds even in light of the fact that the value of Aw would be smaller for higher w in REINFORCE. This analysis suggests that having animals initially perform poorly on the task, even below chance level, could potentially provide more information for learning rule identification.
yields the lowest error, while wi = 2 leads
33

---

A
Wbias
Wbias = true
Wbias 1 true
true
Faster ground truth learning rates
AW
0.35
DW
AW
0.00
0
2
3
.1
0
Stimulus
Stimulus
Stimulus S
Wbias -1
Wbias
Wbias 1
true
true
true
Slower ground truth
0.30
learning rate
AW
AW
AW
Stimulus
Stimulus
Stimulus S
fitted
fitted
fitted
0.30
Fitted
AW
AW
AW
Stimulus
Stimulus
Stimulus
B
W
W
W
0.2
0.2
0.2
0.1
Fist 10us irvesentee Honecc current
Not trab coddunnmaanddo incenet
Fast tdea urrsertied cumert curret
Nut rish unrowarded cument cament
0.0
Pedt 11B resorded comtec Curnec
unzovorrid
unsoeanded incomes paree
-0.1
Nat 3 inbb unspoadaa excred curnare
Pail unsoondal comer
-0.1
-0.1
4.2
Figure 11: Pooling with mixed weight update functions. (A) Following the plotting convention in Fig. 2 we simulate mixed population where half the animals learn with REINFORCE using a higher learning rate (top row) and the other half with a lower learning rate (middle row). The fitted function (bottom row) recovers intermediate update rule that lies between the subpopulations. (B) To test whether the reward-history dependence observed in Fig. is merely a result of faster-learning animals in the pool contributing more recently rewarded trials, we train RNNGLM with mixed learning rates. Reward-condditoneed weight update plots where past trial rewarded (black) and unrewarded (red) do not consistently show the reward history dependence in this setting. This suggests that the history dependence learned by RNNGLM in IBL data is not merely due faster learners receiving more recent rewards; this makes sense as in both this plot and Fig. [ the learning stage is controlled by fixing the stimulus weight w.
a population
34

---

Noise Level
RMSE (Reconstructed Áw vs. Ground Truth)
0
0.0216
70
0.0349
Ia
0.0652
Table 4: Reconstruction accuracy degrades with increasing white Gaussian noise in simulated data. We add zero-mean white Gaussian noise to the ground truth REINFORCE learning rule for simulated data, with the noise standard deviation expressed as fraction of the learning rate a. In other words, the ground truth weight changes are now driven by two components: a learning component and noise component, in [9], whereas our fitted model remains deterministic. We report RMSE between the reconstructed and true Aw. As expected, higher noise leads to worse recovery accuracy.
Wo correctness
RMSE
Default
0.0130
+0.5 perturb
0.0270
+ +1.0 perturb
0.0453
+2.0 perturb
0.1519
Table Effect of initial weight Wo correctness on reconstruction accuracy in simulated data. We test the following scenarios: default, and wo offset by +0.5, +1.0, +2.0 after the estimation. Deviations from the true wo can substantially degrade the accuracy of learning rule recovery, underscoring the importance of accurately estimating Wo-
Additional Input Dim
Default Sample Size
3 Samples
1
1.00
0.59
3
1.09
0.62
10
1.35
0.7
30
1.58
1.20
100
X
1.67
Table 6: Recovery performance (relative RMSE) of the inferred learning rule across increasing input dimensionality. Each row corresponds to input dimensionality setting; additional inputs are distractor streams added the default task in Fig [ An "X" marks failure to capture the qualitative features in Fig. [ features remain captured at higher-D task settings, and reconstruction accuracy improves with more data. Relative RMSE is normalized the baseline (1 additional input, default sample size).
Condition
Relative RMSE
Decision model known
1.00
Model mismatch
1.62
Table Performance degradation with decision model mismatch. We examine a multiplicative decision structure the ground-truth model. The first row corresponds to the where the inference framework knows the true multiplicative decision structure, logit( (rright) = wn c. R2 + wn and context variable between 0 and 1, and leftward cumulative
c) RR. for rightward cumulative cue between 0 and -1. The second row represents model mismatch, where the inference framework still assumes GLM model, ((rii()())))))) wt JT ~UR JR + ú. c.
35

---

A
B
AWstim
AWstim
0.00
0.00
-0.05
-0.5
0.5
1.0
-1.0
0.0
1.0
Stimulus s
Stimulus
Figure 12: Robustness of inferred learning rules across random seeds. We ran all inference procedures with three different random seeds, each achieving similar test log-likelihoode and show the mean standard deviation across seeds as shaded regions. (A) follows the plotting convention of Fig- 5A, showing stimulus- and choice-dependent weight updates. (B) follows the plotting convention of Fig. 5B, showing reward-history-codiiiee
updates. While some variation as side bias, negative baseline, and reward-history
is present, the essential qualitative dependence-art consistently recovered across seeds.
A
B
Watim = 0
Wittm
Watim = 0
0.06
AWstim
AWsstmm
AWssim
0.00
0.00
0.00
-0.05
-3.05
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.5
1.0
-1.0
-0.5
0.0
0.5
Stimulus
Stimulus
Stimulus
Figure 13: Longer history effects observed in the model. In Fig. consecutive rewarded or unrewarded trials, since consecutive trials risk extrapolation. Here we bypass this issue by still conditioning on four consecutive from further back in the trial history, to whether the model can capture dependencies beyond the immediate past. (A) conditioned on trials 7 to 10 trials ago, (B) 17 to 20 trials ago, and (C) 27-30 trials ago. History effects still detectable beyond just four trials, but decay with distance. Plotting conventions are the same as in Fig. 5
53. we conditioned on to four
hnwee/////dwreeddd trials, but taken
36

---

Representing Aw with NN
Representing w with NN
A
On simulated data
Wsttin
Wstim
fitted
fitted
true
200 Trials
400
200
400
Trials
B
On simulated data, 10x more data than in A)
fitted
Wstmm
0-
Wistim
2-
fitted
true
200 Trials
400
200
400
Trials
C
On IBL data
Weights
Weights
Wetim
Whits
2000
4200
6000
8000
10000
2000
4000
6000
8000
10000
Trial
Trial 4
Figure 14: Parameterizing Aw directly (as opposed to w) improves weight trajectory recovery. (A) On simulated data, using a neural network to directly represent the weight update Aw yields more accurate recovery of the weight trajectory compared directly parameterizing w using an RNN. The RNN-based w trajectory is overly smooth, likely due to implicit regularization that biases the model toward simpler solutions in the weight trajectory rather than the weight update function; in contrast, regularizing the weight changes (via Aw) provides a more appropriate inductive bias (see Discussion).
(B) As expected, the effect of implicit regularization is more pronounced in underconstrained settings (e.g., low-data regimes), and becomes less critical as more data constraints are introduced; since with sufficient data, both models should approximate the learning trajectory well as per the universal approximation results. (C) On IBI. data, this distinction is also evident: when training on just one example animal, parameterizing A leads more realistic weight trajectories than using an RNN to represent directly.
37

---

CSHL,002
CSHL_003
Pryfoccc
10.00
CSHIL000
007
100
Trok CSHL,000
CSHIL 002
CSHILO1A
CSHLO11
Vibes
Figure 15: Extended version of Fig. 4A showing both stimulus and bias weight trajectories across more animals. The decision weights recovered by DNNGLM closely resemble those inferred by PsyTrack.
CSHL 001
CSHL DC2
CSAL 003
CSHL DDM
REAFORCE Porreg
23RFORCE noeees
SCMFORCE NOmee!
CSHL 005
CSHL 006
CSHL
Pyytic
CSHL OLD
CSHL 012
CSHL 014
CSHtL 015
ww
4.00
Figure 16: Similar to Supp. Fig. 15 but showing the decision weights recovered by REINFORCE (with the nonnegative baseline constraint), which was previously shown to yield worse log-likelihoods (Tables I and 2). We see here that it leads to greater discrepancies from the weights inferred by PsyTrack.
38

---

Wasli
Walim
Wistim 2
AWblas
AWbies
AWbias
0.0
0.5
-10
-0.5
-0.5
0.0 Stimulus
0.5
1.0
Stimulus
Stimulus
Figure 17: The inferred update function for the bias weight, using RNNGLM, qualitatively matches the fitting in Fig.
COAALOOE
Figure 18: Similar to Fig. 15 but for RNNGLM, where recovered weights also closely resemble those inferred by Psy Track.
39


Reviews:
[{"review_id": "nycIw5U9sZ", "reviewer": "Reviewer_DYot", "summary": "This paper looks at how animals learn new tasks from scratch. The authors argue that previous methods for modeling animal learning are too restrictive. They either assume a specific, predefined mathematical form for the learning rule (like Q-learning or REINFORCE) or are limited to simple bandit tasks where the underlying task structure is already known.\n\nThe main subject of this paper is that it proposes a flexible new framework that uses deep neural networks to infer the learning rule directly from an animal's behavioral data. The goal is to discover the actual and potentially complex strategies animals use without being constrained by human assumptions.", "questions": "1. Could you clarify the nature of the error bars in Figure 4B (Standard Deviation? Standard Error of the Mean over animals? Over cross-validation seeds?)?\n\n\n2. The current model assumes the learning rule f_θ is static throughout the entire course of learning. However, it's plausible that animals \"learn how to learn\". Is it possible that the learning rule itself is non-stationary and adapts as the animal gains more experience with the task structure?\n\n\n3 Could you perform an analysis where you fit your RNNGLM independently to the first half and the second half of the sessions for each animal? If the inferred f_θ is qualitatively different between the early and late phases of learning, this could be strong evidence for a non-stationary learning rule that the current model cannot capture.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength\n\nThe framework demonstrated its accuracy in simulation. The DNNGLM successfully recovered the weight trajectories and underlying known Markovian learning rule. Crucially, when faced with a non-Markovian rule, the DNNGLM failed, while the RNNGLM successfully captured the history-dependent effects, consistent with the author’s claim for the necessity of the recurrent architecture for complex learning.\n\nWeakness\n\nThe choice of a GRU is perhaps a limitation since the field has largely moved on toward transformer-based architectures given their ability to do long-range dependencies. The paper discuss the use of Transformers to a single sentence in the future directions section.\n\n The non-Markovian effects reported span just four trials (Fig. 5B). Is this a genuine feature of mouse learning, or is it a limitation imposed by the GRU's difficulty in propagating information over longer horizons? A Transformer, with its direct attention mechanism, could potentially discover much longer temporal dependencies, and could lend even more weight to the authors’ conclusions.", "was_revised": true, "final_justification": "I thank the authors for their detailed reply. I am increasing my score to 4.", "rebuttal": "We are very grateful to the reviewer for their detailed and insightful comments. We especially appreciate the suggestions regarding model architecture, temporal structure, and nonstationarity, which helped us expand and clarify several key aspects of our work. Below, we summarize how your feedback led to specific improvements in our analyses and manuscript.\n\n**Transformer extension**. We strongly agree that extending to Transformer-based models is an important next step, given the known limitations of GRUs. Motivated by the reviewer’s suggestion, we implemented a Transformer-based parameterization of the weight update function. As shown in **Table A** below, it achieves modestly improved performance. Also, the inferred learning rule plot, which we'll add to the manuscript, captures non-Markovian reward history effects (stronger weight updates after consecutive rewards, similar to RNNGLM). These findings further underscore the history-dependent characteristics we uncover and highlight the promise of Transformer-style models for more in-depth future work, particularly in the larger-data settings envisioned for future extensions. We will add these results and corresponding discussion to the revised manuscript. \n\n| Model          | Test per-trial LL | P-val (comparing to RNNGLM) |\n|----------------|-------------------|------------------------------|\n| RNNGLM         | -0.4837           | n/a                          |\n| TransformerGLM | -0.4806           | 9.72e-03                     |\n\n*Table A*: Comparison of RNNGLM and TransformerGLM test performance on the IBL dataset.\n\nWe also hope to clarify that the GRUs were chosen as a practical and interpretable starting point for de novo learning rule inference (see our discussion with CMVq, under “Novelty”, for more details). GRUs offer favorable inductive biases, which are especially useful in behavior-level modeling with limited and noisy data. In particular, MLPs — substructure in GRUs  — are known to exhibit spectral bias, favoring smoother, low-frequency functions during training [70-82]; this inductive bias is well-characterized and can be directly tuned (e.g., via width), which can promote generalization even in data-limited regimes. Also, our new analysis in the next paragraph indicates that GRUs can capture dependencies beyond just a few trials. In contrast, Transformer-based models are more expressive but have less well-understood inductive biases; that said, recent work (e.g., Vasudeva et al., ICLR’25) shows that Transformers can also exhibit simplicity biases under certain training conditions. \n\n**Why illustrated Non-Markovian effects over only four trials**. Thank you for raising this thoughtful question. The four-trial horizon in Fig. 5B is not meant to imply that GRUs can’t learn further than four trials, but a pragmatic choice in illustration (we will clarify this in the updated figure legend). In our dataset, long sequences of consecutive unrewarded trials (e.g., >5 in a row) are rare, making it difficult to estimate such effects without extrapolating beyond the fitted data. To further probe long-range dependencies, we analyzed $\\Delta W$ conditioned on reward outcomes from ten trials ago—specifically comparing sequences in which all four consecutive trials from eleven to eight steps prior were rewarded versus unrewarded. We observed a consistent separation in $\\Delta W$ across these conditions, suggesting that the model captures non-Markovian effects extending well beyond a few trials. We will add this plot and clarification to the revised manuscript, and could include additional analyses to quantify how far back in time these differences remain detectable. Our goal was to highlight clear evidence of non-Markovian reward-history structure using an illustrative example as a step beyond conventional Markovian models. \n\nPrompted by the reviewer’s suggestion, we ran additional analyses truncating the history window. As shown in **Table B below**, performance degraded when history was limited to 10 trials, confirming the importance of long-range dependencies. We also ran a new simulation with a handcrafted longer-horizon ground truth learning rule to show that RNNGLM can effectively capture influences well beyond a few trials. Both the new simulation plot and the following table will be included in the revised manuscript to underscore the importance of modeling extended history-dependencies. \n\n| Model                                | Test per-trial LL | P-val (comparing to no truncation) |\n|--------------------------------------|-------------------|------------------------------------|\n| TransformerGLM (with truncated memory) | -0.4838          | 1.44e-03                           |\n\n*Table B*: Impact of history truncation (after 10 trials) on test log-likelihood in the IBL dataset. We compared model performance with and without memory truncation (i.e., restricting context length) to assess the importance of extended temporal time window. \n\n**Clarify nature of the error bars**. Thank you for catching this omission. The error bars reflect standard deviation across cross-validation seeds. We will clarify this in the updated figure legends. \n\n**The nonstationarity extension**. This is a great question. As noted in the manuscript, our current model assumes a static $f_\\theta$, consistent with common practice. To probe potential nonstationarity in the learning rule, we followed the reviewer’s suggestion and fit separate models to early and late learning phases. However, we did not observe significant improvement on test data (p = 9.98e-01); and while the fitted learning rules differed slightly, core qualitative features (e.g., reward-history dependence and asymmetry) remained consistent. We also tried adding trial numbers as an auxiliary input, again with no performance gain. These results suggest stable trial-level learning patterns in this dataset—though we agree that more explicit modeling of dynamic rules is a promising future direction (e.g. via phased fitting as the reviewer suggested or adding trial number auxiliary input). We thank the reviewer for raising this and will include these results and discussion in the revision."}, {"review_id": "IFjIG53RAN", "reviewer": "Reviewer_CMVq", "summary": "This paper proposes a method to meta-learn algorithmic-level learning rules from binary decision-making data during learning. The authors compare two learning rules parameterizations. Both learning rules output updates to the weights of a dynamic Bernoulli GLM (the policy). The learning rules’ inputs are the current stimulus, task covariates, current learning rule parameters and reward. They compare a feedforward neural network (DNNGLM), which is Markovian in the sense that only the current trial information is used to compute the changes to the policy network, and a recurrent network (RNNGLM), which is non Markovian as it can keep previous trial information in its hidden neuron dynamics.\n\nThey first verify the ability of their method to recover a ground-truth learning rule on an in silico dataset, before inferring learning rules on an IBL dataset. On this dataset, they find that RNNGLM can fit the data better than DNNGLM, thus providing evidence for non-Markovian learning in rodents. They reverse engineer the successful learning rules, which display interesting features such as negative baseline and side asymmetries.", "questions": "1/ Novelty: The authors insist on the “de novo” learning aspect, but this seems to me more of a dataset choice than a limitations of previous methods. Could you elaborate? \n\n2/ Degeneracy: how robust are the conclusions made in this study, specifically for the IBL dataset? I saw in supplementary what appeared to be a thorough robustness analysis of the simulated data case, but not for the IBL data case, but maybe I’m mistaken. What are the error bars in fig 4B? If the robustness analysis that I’m asking for is already present in the paper, I am a bit surprised by how similar all the recovered learning rules look, are there differences between them?\n\n3/ Is the comparison of DNNGLM to RNNGLM a fair one, doesn’t the RNN have more parameters (the hidden state)/be a generalization of the DNNGLM and would therefore be expected to fit data at least as well than DNNGLM? Is this point covered by the DNNGLM with the history of previous trials?\n\n4/ L78: on the disadvantages of synaptic level learning rules: these types of learning rules can and have been fit to behavioral data in the references already cited in this manuscript, they are not restricted to neural data. I think that besides having similar names (learning rules and plasticity/learning rules), these rules have little in common, and there is no need for the authors to further differentiate themselves from them.\n\n5/ L265: “recovering mechanistic insights”. I don’t understand what mechanisms are being uncovered in this study, the learning rules considered operate at an abstract level: the weights are policy weights, not actual synapses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- The paper is clearly written and presented, with a thorough literature review.\n- The methods and most of the results appear sound and trustworthy to me.\n- The paper goes beyond proposing a novel method and testing it on toy tasks, by considering a real dataset and makes actual predictions for learning strategies in mice. Note that I don’t feel knowledgeable enough to assess the novelty/impact of these results on interpreting the inferred learning rules (non-Markovian, side asymmetry and negative baseline).\n\n\nWeaknesses:\n\n- Novelty (from the method point of view): This work positions itself in a crowded subfield of meta-learning learning rules that has been very active in the past 5+ years, and I am slightly hesitant to label the novelties of the method highlighted by the authors as properly novel rather than variations on previous methods (see question 1 below).\n- Degeneracy: I would expect many different learning rules to fit the binary decision data well, i.e. your model should be quite under-constrained by the experimental data at hand, and I would thus expect many different learning rules to perform well (see question 2 below).", "was_revised": true, "final_justification": "The authors have made meaningful improvements in response to my (and other reviewers’) comments, adding clarifications and analyses that undeniably strengthen the paper. However, while the method is novel, technically sound, and clearly presented, I still have reservations about this paper's potential impact and distinctiveness in an already crowded subfield. I therefore maintain my score of 4 and recommend borderline acceptance for NeurIPS.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of our work’s clarity, soundness, and relevance to understanding learning strategies from real data. Your comments helped us substantially improve the manuscript by clarifying our contributions and novelty, as well as adding new analyses to address concerns about degeneracy and fairness. Below, we respond point-by-point to describe these improvements and how your suggestions strengthened the paper. \n\n**Novelty**. We thank the reviewer for this important comment. In particular, we appreciate the opportunity to clarify why the *de novo* framing represents more than a dataset choice — it reflects a fundamentally different modeling challenge. Most prior meta-learning work focuses on training agents to learn well across tasks, often assuming normative objectives (e.g., reward maximization). These approaches are designed to improve task performance, not to recover the actual learning rules employed by an agent (e.g., an animal). \n\nIn contrast, our work addresses **a fundamentally different challenge: inferring the learning rule that a biological agent used to learn from scratch**. This de novo setting presents several new challenges. First, animals often behave in \"suboptimal\" ways that would not appear to increase their expected reward (Akrami et al., Nature, 2018; Kastner et al., Cell reports, 2022), which violates the normative assumption. Without this assumption, the solution space becomes far less constrained—animals may exhibit complex and surprising learning dynamics, including history-dependence, biases, or inconsistent strategies. Capturing such richness requires flexible models that do not rely on strong structural priors. Second, external stimuli (not just reward histories) must be integrated into the update, which introduces complexity and goes beyond classic multi-armed bandit models. Third, many existing approaches assume fixed feature representations, static policy forms, or hand-designed learning rules that do not adapt during training — limiting their flexibility. These challenges limit the applicability of previous methods, which achieve lower performance compared to ours (Table 1 in the manuscript). \n\nWe will revise Related Work and Discussion to better highlight this distinction and clarify the methodological and conceptual novelty of our contribution. \n\n**Degeneracy**. We agree that inferring flexible learning rules from behavior poses fundamental challenges around degeneracy—an inherent issue in this domain, also encountered in related work such as [58]. Indeed, running our models with different random seeds yields slight variations in recovered rules, despite similar train and test log-likelihoods. To quantify this, we repeated all inference runs across multiple seeds and added shaded regions denoting standard deviation to the learning rule plots. **While some variation exists, the key interpretable features—such as side bias, negative baseline, and reward history dependencies—are consistent across seeds**. We will include these plots in the updated manuscript. \n\nImportantly, these features are not merely visual artifacts but functionally meaningful: eliminating the negative baseline via a non-negativity constraint significantly reduced performance (Table 1 in the manuscript), and so did truncating history (**Table B** in response to Reviewer DYot). Additionally, our method recovers ground-truth rules in simulation, further demonstrating its ability to identify meaningful structure despite under-constrained settings. \n\nWe initially omitted shaded variability regions to highlight overall trends, which appeared stable across seeds. However, we appreciate the reviewer’s point and will revise all plots to reflect variability—further underscoring that the key trends are robust and exceed the variation. We will also expand the limitations section to explicitly discuss degeneracy and highlight these new plots illustrating robustness. We will also update Fig. 4B figure legend to explain that the error bars reflect standard deviation across cross-validation seeds; we apologize for the omission. \n\nFinally, we would like to clarify that our work has so far incorporated several mitigation strategies to help address degeneracy, which we will more clearly highlight in the revised Discussion. First, we used cross-validation to assess generalization. Second, we included structural ablations and constraints (e.g., non-negativity, history truncation) to test the necessity of key features. Third, we leveraged large-scale behavioral data, where extended training over thousands of trials is common, and pooling across animals increases data availability. We will expand our Discussion section to more clearly articulate these points. \n\n**Is Comparison of DNNGLM to RNNGLM fair**. We appreciate the reviewer’s concern and agree that RNNGLMs have more parameters due to their hidden states. To test whether the performance gap stems purely from model capacity, we conducted additional comparisons where we matched the parameter count between DNNGLM and RNNGLM. Even with equalized capacity, RNNGLM continued to outperform DNNGLM (p = 5.53e-04), suggesting that the key driver of the gap is not the model size. Importantly, no matter how large a DNN becomes, its memoryless architecture prevents it from capturing history dependencies—unless history is manually appended as input, which introduces brittleness and hand-tuning. We will add these new results and clarifications in the revised manuscript. \n\n**On synaptic level learning rules**. We agree with the reviewer that synaptic-level learning rules and the behavioral-level rules we study operate at fundamentally different levels of abstraction, and thus have little in common beyond the shared term “learning rule.” To avoid confusion, we will simply cite these synaptic plasticity studies as a complementary line of work—focused on neural-level mechanisms that can also, in some cases, be fit to behavioral data—and remove all subsequent unnecessary sentences that attempt to further distinguish our approach.\n\n**Mechanistic insights**. We agree that “mechanistic” may be misleading here. Our intent was to highlight interpretable structure in the inferred learning rules (e.g., asymmetries, biases), not biophysical mechanisms. We will revise the manuscript to remove the word “mechanistic”. \n\n**Broader outlook**. Before closing, we’d like to note that the reviewer’s comment, on novelty and de novo learning, has prompted us to reflect more deeply on the broader vision and significance of this work.\n\nDe novo learning—where agents must learn from scratch in a novel task without strong priors—is especially **important in neuroscience, where animals are often trained in entirely new tasks**. Understanding how learning unfolds during this phase is both scientifically valuable and practically important for improving training protocols and experimental design. Yet despite its ubiquity, this regime remains understudied. \n\nWe also note that learning rule inference is gaining traction, with recent works (e.g., [21, 24, 46, 58]) exploring related questions in structured learning and agent modeling—though not on the de novo setting we consider here. This makes our focus on inferring learning rules from scratch particularly timely for NeurIPS 2025, as the field begins to tackle more open-ended and biologically grounded formulations of learning. If the reviewer is aware of especially relevant prior work, we would be genuinely happy to engage further.\n\nBy reflecting on the reviewer’s feedback, we believe our approach lays a foundation for a broader research direction. While our work is the first to infer nonparametric, non-Markovian learning rules in de novo settings, we see this as just the beginning. Animals in naturalistic environments face even more complex challenges that require flexible strategies constructed from scratch [14]. Our approach offers a principled first step toward modeling such capabilities, and we hope it inspires future work that builds on this foundation using richer tasks, additional behavioral signals, and more expressive architectures. \n\nWe will revise the manuscript to more clearly highlight this broader framing in the Related Work and Discussion sections."}, {"review_id": "1k2Xd6Nfi2", "reviewer": "Reviewer_9hyV", "summary": "In this paper, the authors study propose methods for inferring learning rules underlying behavior in de novo learning tasks. In particular, they use neural networks (both feedforward and recurrent) to parameterize changes to input weights over time. They apply their method to simulated data as well as the IBL dataset, and demonstrate that recurrent architectures succeed in inferring non-Markovian learning rules, and also outperform methods like REINFORCE.", "questions": "- Regarding Fig. 3, the RNN GLM appears to miss the weak dependence of $\\Delta W_{stim}$ on the stimulus $s$, something that is attributed to finite data effects. Why should finite data be a noticeable problem for this task considering that it is with simulated data?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is well-written and the methods are clearly explained and intuitive.\n- The proposed methods reveal interesting learning rule phenomenology (e.g. non-Markovian, asymmetric, suboptimal negative baseline) from real data that is not captured by existing methods like REINFORCE. \n\nWeaknesses:\n- As far as I can tell, across both the simulated and real data experiments, only tasks with 1D stimuli are used. This is understandable in the context of the IBL dataset, but it is unclear whether these methods do well beyond this simplified setting, given possible concerns about sample complexity (as illustrated in Fig. 2c). \n- The choice of modeling the animal's policy as a GLM seems prohibitively restrictive in general, albeit it does aid in interpretability of learning rules, and does seem adequate for the IBL dataset based on Table 1. Similar to the previous point, I would imagine this modeling choice would be problematic for any task with multiple stimuli that, e.g., are selectively attended to in a context-dependent manner? If so, this concern limits the applicability of the proposed method.", "was_revised": true, "final_justification": "The additional experiments have clarified some of my concerns about the scope of the proposed method. As such, I am increasing my score to 5.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of our manuscript’s clarity and the interesting learning rule phenomenology uncovered. Your suggestions (particularly regarding scope and generalization) helped us expand the manuscript with new analyses, clarify our framing and contributions, and better highlight the broader applicability of our approach. Below, we address your points in detail and summarize the corresponding improvements to the paper. \n\n**Extension to higher-D input and better clarification of scope & significance**. We thank the reviewer for highlighting the importance of extending our approach to higher-dimensional and more complex learning settings. To evaluate generalization to higher-D inputs, we increased the input dimensionality by adding additional distractor input streams (results summarized in **Table C** below). We also tested a 2D version of the Poisson click task with context modulation (see the next section). We acknowledge that extending to higher-dimensional inputs increases sample complexity, as expected from the curse of dimensionality. Nonetheless, our experiments show that even with higher-D inputs, key behavioral features are recoverable (**Table C**)—and with more data, recovery quality improves substantially. To substantially strengthen the manuscript by demonstrating robustness beyond the 1D setting, we'll include these results (**Table C** and fitted function plots) and this discussion in the updated manuscript version. \n\n| Additional Input Dim | Default (Relative RMSE) | Features Captured | 3× Samples (Relative RMSE) | Features Captured |\n|----------------------|--------------------------|--------------------|-----------------------------|--------------------|\n| 1                    | 1.00                     | ✓✓✓                | 0.59                        | ✓✓✓                |\n| 3                    | 1.09                     | ✓✓✓                | 0.62                        | ✓✓✓                |\n| 10                   | 1.35                     | ✓✓✓                | 0.75                        | ✓✓✓                |\n| 30                   | 1.58                     | ✓✓✓                | 1.20                        | ✓✓✓                |\n| 100                  | -                        | -                  | 1.67                        | ✓✓✓                |\n\n*Table C*: Recovery performance of the inferred learning rule across increasing input dimensionality. The additional input streams follow the same distribution as the original input. Checkmarks (✓) indicate the model successfully captured key qualitative features from Fig. 2: (1) slowing at high weights, (2) amplitude-dependent learning, and (3) choice asymmetry; – indicates failure to capture these features; RMSE computed on normalized to the baseline (1D, default sample size).\n\n*To better clarify the scope of our original submission*: as the reviewer also acknowledged, we focused on simplified 1D tasks because they mirror the IBL datasets and many laboratory neuroscience settings where animals are introduced to novel tasks and must learn from scratch. This de novo learning scenario—where prior structure or instruction is minimal—is both conceptually significant and common in experimental contexts (see also discussion with CMVq under “Novelty”). \n\nEven this simple task setting involves several key hallmarks of biological learning—e.g., history dependence, biases, and suboptimal strategies—and provides a tractable platform for reverse-engineering learning dynamics. As in prior foundational work (e.g., XOR, Poisson clicks [Brunton et al., Science, 2013]), this ‘start simple, validate, then generalize’ strategy allows us to isolate core principles before scaling up. Our new results (**Table C**) demonstrate that this approach can extend to higher-D inputs, indicating its broader applicability and motivating future work. We will add a paragraph reflecting these discussion points to the manuscript. \n\n**GLM too restrictive**. We thank the reviewer for this insightful point, which prompted us to run new simulations and better delineate the scope and limits of our current framework. While this choice of GLM aids interpretability and aligns with common practice in neuroscience, we acknowledge that it limits expressivity and will highlight this limitation more explicitly in the revised manuscript. \n\nTo probe this concern, we tested a more complex 2D version of the Poisson clicks task with context-dependent decisions, where the relevant stimulus dimension is modulated multiplicatively by a context variable (**Table D** below). This task introduces a nonlinear decision mapping that violates GLM assumptions. To better understand the limits of our method, we first evaluated the idealized case where the true decision mapping is known to the inference framework—a setting not assumed to reflect real-world use but helpful for benchmarking inference quality. As expected, our method accurately recovers the learning rule in this case. Under model mismatch—where the inference framework assumes a standard GLM instead—performance degrades, but the model can still capture key qualitative features of the learning rule. Both the robustness and limitations of our current framework are illustrated by these results and the clarified scope discussion, which will be incorporated into the revised manuscript. \n\n|                          | Relative RMSE | Features Captured |\n|--------------------------|---------------|--------------------|\n| Decision model known     | 1.0           | ✓✓✓                |\n| Model mismatch           | 1.62          | ✓✓✓                |\n\n*Table D*: Recovery of key qualitative features (in Fig. 2) in a context-modulated Poisson click task. The first row corresponds to the case where the inference framework knows true decision structure, $logit(P(\\text{right})) = w_L \\cdot c \\cdot x_L + w_R \\cdot (1 - c) \\cdot x_R$ for rightward cumulative cue and context variable $c$ between 0 and 1, and leftward cumulative cue between 0 and –1. The second row represents model mismatch where the inference framework still assumes a GLM model  $logit(P(\\text{right})) = \\hat{w}_L \\cdot x_L + \\hat{w}_R \\cdot x_R + \\hat{w}_c \\cdot c$.  ✓ = feature captured; RMSE relative to known-model case.\n\n**About Figure 3**. We thank the reviewer for the great question. While the black curve (conditioned on past several consecutive correct trials) clearly shows stimulus dependency, the red curve (conditioned on past several consecutive incorrect trials) shows weaker effects. This is primarily due to finite data, which applies even in our simulations: our simulated animals learn relatively quickly, resulting in few instances of consecutive errors. This imbalance makes it harder for the model to learn the error-conditioned cases due to sparser coverage. To test this explanation, we simulated with animals starting at worse performance, thereby increasing the number of consecutive incorrect trials. As expected, the model then recovered the dependency more reliably. This observation also connects to the Fisher Information analysis in Appendix A, which highlights how various experimental variables (e.g. initial condition) can affect inference, and that knowledge could help to inform experimental design. We will include these new plots and a paragraph reflecting this discussion in the revised manuscript. \n\n**Broader Framing and Impact**. We appreciate the reviewer’s comments, which prompted us to reflect more deeply on the broader vision and significance of this work.\n\nOur study is especially timely for the NeurIPS community, which is increasingly engaging with problems at the intersection of deep learning and neuroscience. As the literature on inferring biological learning rules continues to grow (Related Works), a central and still underexplored challenge is to model de novo learning settings, where prior structure is minimal and animals must learn from scratch. These settings demand models that are both flexible enough to capture complex, history-dependent behavior and interpretable enough to yield scientific insights.\n\nOur framework directly addresses this need. By fixing the decision model using standard, well-supported forms, we isolate the inference of trial-by-trial weight updates—an underexplored but foundational problem. This modular design mirrors a common strategy in both neuroscience and machine learning, where complex systems are understood by studying components separately before tackling joint inference (e.g., [46]). Our results show that even within this simplified setup, rich behavioral features can be recovered and generalized to more complex settings (e.g., higher-dimensional inputs and context-dependent tasks), as we now demonstrate in our revised manuscript.\n\nThis framework thus provides a new lens on suboptimal, history-dependent learning in animals and lays the groundwork for future extensions—such as jointly inferring both learning rules and policy mappings—once additional data or stronger inductive biases are available. Motivated by the reviewer’s feedback, we have added new experiments and clarified these broader implications in the manuscript to better articulate how this work serves as a stepping stone toward modeling more complex and ecologically valid settings of biological learning. We will revise the manuscript to more clearly highlight this broader framing."}]

Author rebuttal:

Meta-review: This paper proposes a flexible framework for inferring learning rules directly from animal behavior in de novo learning tasks. Rather than assuming a parametric form like Q-learning or policy gradient, the authors parameterize the learning rule with neural networks, allowing per-trial policy updates to be learned from data. They validate the approach on simulations, where the method recovers known rules, and then apply it to a large mouse dataset. Results show that recurrent parameterizations capture history-dependent, non-Markovian learning dynamics, and reveal asymmetries between correct and incorrect trials. The rebuttal further strengthened the paper by adding Transformer-based extensions, long-horizon analyses, and checks for nonstationarity.

Reviewers agreed that the paper is clearly written, technically sound, and provides a useful framework for reverse-engineering learning rules from behavior. Strengths include the novelty of applying flexible neural parameterizations to de novo learning, the combination of simulation and real animal data, and interpretable findings such as side biases and negative baselines. Reviewers also appreciated that the authors went beyond toy problems to extract insights from real data, and that additional experiments addressed concerns about scope, degeneracy, and architecture choices.

The main weakness concerns were about novelty in a crowded subfield, potential degeneracy in inferred rules, and limits of the modeling choices (GLM decision policy, GRUs vs Transformers, 1D tasks). Some reviewers felt the contribution is incremental relative to prior meta-learning work, and that applicability to richer tasks remains to be shown. Still, the rebuttal added convincing evidence that key behavioral features are robust, clarified limitations, and positioned the work well within ongoing efforts in computational neuroscience. On balance, this is a timely and solid contribution that moves the field forward. I recommend accept (poster).
Decision: Accept (poster)