Volume Transmission Implements Context Factorization to Target Online Credit Assignment and Enable Compositional Generalization
Authors: Matthew Storm Bull
Po-Chen Kuo
Andrew Lawrence Smith
Michael A Buice
Venue: NeurIPS 2025 poster ()
Keywords: Meta-learning
Bio-inspired
Hypernetworks
Compositionality
Sequence Learning
Primary area: neuroscience_and_cognitive_science
TL;DR: The volume transmission of small molecules implements a context-factorized hypernetwork capable of online credit assignment and compositional learning reminiscent of neurobiology.

Abstract: The modern connectivist framing of neural computation emphasizes the primacy of synaptic communication at the risk of neglecting the influence of the surrounding neuromodulatory environment --- a neuron's 'biophysical context.' Decades of experimental work has established two views of neuromodulatory (NMs) influence: 1) NMs significantly alter circuit dynamics and 2) NMs gate synaptic plasticity, acting as a 'third factor' in learning. Here, we unify these perspectives, proposing that neuromodulation via volume transmission implements a powerful computational principle: context factorization. We derive an endogenously neuromodulated Recurrent Neural Network (e-nmRNN) from a rate reduction of NM release, showing how NM concentrations dynamically factorize network connectivity. This framework reveals how multiplicative NM gating distinctly influences dynamical regimes compared to additive input. Crucially, this context factorization enables targeted online credit assignment: learning rules derived for the e-nmRNN are naturally gated by NM concentrations, localizing updates to relevant contexts. We demonstrate that e-nmRNN dynamics can learn to approximate gradient descent, facilitating rapid in-context adaptation akin to meta-learning. Empirically, e-nmRNNs achieve strong compositional generalization in sequence-to-sequence tasks, outperforming baselines and exhibiting greater hyperparameter robustness. Furthermore, when trained on complex multitasking benchmarks, e-nmRNNs develop emergent properties mirroring biological observations, including modularity, cell-type specialization based on NM release, and distinct neuromodulatory timescales encoding task context. The model's interpretability allows us to reverse engineer these emergent structures. Notably, in reinforcement learning tasks, the e-nmRNN learns to encode context and signals like Reward Prediction Error (RPE) within its neuromodulator dynamics, demonstrating a mechanism for RPE-gated online credit assignment essential for learning how to learn. By bridging biophysical mechanisms with computational principles and empirical validation, our work presents e-nmRNNs as a performant, interpretable model for understanding the computational role of neuromodulation in flexible and compositional learning.

Volume Transmission Implements Context Factorization to Target Online Credit Assignment and Enable Compositional Generalization
Matthew S. Bull
Po-Chen Kuo, Graduate Program in Neuroscience University of Washington Seattle, WA 98109
Shanahan Family Foundation Fellow
Allen Institute + University of Washington
Seattle, WA 98109
matthew.C matthew.bullie wuw.aaalelniiisiitteee org
Andrew L. Smith, Shanahan Family PostBac Fellow
Michael A. Buice
Allen Institute + University of Washington
Allen Institute + University of Washington Seattle, WA 98109
Seattle, WA 98109
wachaalbuulllleennstttttuttt org
Abstract
The modern connectivist framing of neural computation emphasizes the primacy of synaptic communication the risk of neglecting the influence of the surrounding neuromodulatory environment - a neuron's 's 'biophysical context." Decades of experimental work has established two views of neuromodulatory (NMs) influence: 1) NMs significantly alter circuit dynamics and 2) NMs gate synaptic plasticity, acting as a 'third factor' in learning. Here, we unify these perspectives, proposing that neuromodulation via volume transmission implements a powerful computa- tional principle: context factorization. We derive an endogenously neuromodulated Recurrent Neural Network (e-nmRNN) from rate reduction of NM release, show- ing how NM concentrations dynamically factorize network connectivity. This framework reveals how multiplicative NM gating distinctly influences dynamical regimes compared to additive input. Crucially, this context factorization enables targeted online credit assignment: learning rules derived for the e-nmRNN are naturally gated by NM concentrations, localizing updates to relevant contexts. We demonstrate that e-nmRNN dynamics can learn to approximate gradient de- scent, facilitating rapid in-context adaptation akin to meta- meta-learning. Empirically, e-nmRNNs achieve strong compositional generalization in sequence-to-sequuecee tasks, outperforming baselines and exhibiting greater hyperparameter robustness. Furthermore, when trained complex multitasking benchmarks, e-nmRNNs develop emergent properties mirroring biological observations, including modu- larity, cell-type specialization based on NM release, and distinct neuromodulatory timescales encoding task context. The model's interpretability allows us to reverse engineer these emergent structures. Notably, in reinforcement learning tasks, the e-nmRNN learns to encode context and signals like Reward Prediction Error (RPE) within neuromodulator dynamics, demonstrating a mechanism for RPE- aPE-gated online credit assignment essential for learning how to learn. By bridging biophysi- cal mechanisms with computational principles and empirical validation, our work presents e-nmRNNs as a performant, interpretable model for understanding the computational role neuromodulation flexible and compositional learning.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
The adaptability of biological neural networks relies neuromodulation 1 3
Beyond rapid synaptic transmission, neuromodulators (NMs) influence computation via slower, diffuse volume transmission [5 6 shaping neuron's 'biophysical context." This influences eircuit dynamics 8 9
10] and gates synaptic plasticity
12 13 We propose that volume transmission implements context factorization [14]. computational principle unifying these roles by defining the moment to moment dynamics as product of the dynamics in different contexts and belief about which context most relevant. We hypothesize neural dynamics decompose into a slowly varying context state (NM concentrations) and faster neuronal activity [15 16 17, providing a parsimonious mechanism for dynamic network reconfiguration
4
We formalize this intuition with an endogenously neuromodulated Recurrent Neural Network (e- nmRNN), derived from a rate reduction of biophysical model of NM volume transmission. The "endogenous" nature is its defining feature: the neuromodulatory state is generated by the network's s own recurrent activity, creating closed-loop, self-modulating system. This allows the network to learn to generate its own internal context states, enabling it to autonomously infer context from an input stream and adapt accordingly. The resulting model is related through nonlinear transformation (rate + synaptic drive) to the recent model from Costacurta et al [15]. Direct comparison after transformation reveals two structural changes: 1) Costacurta et al [15] incorporate low-rank modu- latation passing through a second nonlinearity and 2) direct coupling of context and neuromodulation. Our work complements this perspective by directly connecting biophysical parameters to dynamic connectivity.
We view this alternative derivation, resulting in a closely related but non-identical form, as strong support for the value of this model family. The e-nmRNN forms a context-factorized hypernetwork [8]. where the NM state modulates effective synaptic weights, linking attention [2]] state-space models [22] and dynamic networks 23
19
24
Our central claim: context factorization enables targeted online credit assignment and robust com- positional generalization. c-nmRNN learning rules are gated by NM state, directing updates to relevant contexts. NM dynamics can converge to approximate gradient descent effective weights, facilitating in-context adaptation akin to meta-learning [55] 26 20 We conduct two main exper- iments one in supervised learning and one in reinforcement learning-
probe the emergence of gredit-assigmment-ike dynamics in meta-learning environments. We also conduct two further explorations on the facilitating role such learning- to-larr plays for succeeding in compositionally
structured tasks 27 28
Our empirical findings underscore the value of volume transmission implemented context- factorization. We show that e-nmRNNs achieve strong compositional generalization, outperforming baselines. On multitasking benchmarks (27), trained networks show adaptive generalization and emer- gent structures (modularity, cell specialization, timescale separation) mirroring biology. In addition, the model's interpretability allows for setrrobioooogiiispreee reverse engineering, confirming context encoding in NM dynamics. Furthermore, we demonstrate the framework's ' relevance to reinforcement learning, showing how the model learns to encode critical learning signals like Reward Prediction Error within its neuromodulatory state, opening the door to RPE-gated online adaptation.
This work bridges neurobiology and machine learning, offering interpretable architecture with strong inductive biases for flexible, compositional learning for advancing the two-way conversation within neuro-AI (2).
2 Endogenously Neuromodulated RNNs as Context-Factorized Hypernetworks
Here we outline the derivation and connection of these equations to other model classes. For a more in-depth discussion please see Appendix section I and section 2 respectively.
2.1 Derivation from Modulated LIF Neurons
Starting with Leaky Integrate-and-Fire (L.IF) neurons where synaptic efficacy modulated by a "well-mixed" environment of NMs surrounding all neurons 3 4.. This well-mixed assumption is

---

Volume transmission implements context
B
output sequence Narrhhuunthuu
factorization
v(()) Dr(()
Input sequence
11.0
ILEM
I(t)
mixed pool of neuromodulatoors
Sid 02-
Wg
Arass Way
Taakk n 1W
() F(F()
Figure 1: Volume transmission implements context factorization. A) The ambition of this work is to link neurobiological volume transmission of small molecules with powerful computational primitive: context factorization the ability decompose dynamics and context. This approach unifies both targeted online credit assignment with neuronal dynamic changes under different conditions. B) Our tool for connecting substrate with computation is the endogenously neuromodulated RNN characterized by the reciprocal coupling of neuronal rates F(t) and neuromodulator concentrations (t())) as a principled 'next leading order contribution' RNN dynamics.
principled abstraction, interpreted as timescale separation where diffuse NM concentrations establish stable "context" for faster neural dynamics. We codify this to leading order by its contribution the effective synaptic strength W We add matrix Taylor Expansion around the baseline synaptic weights (WT)) in the form of learned tensor Tyyk- This tensor should be interpreted how the effective coupling between neurons and changes in the presence of the kth NM ny (t):
w (t)= W
Σ (t)
Importantly, NM concentrations follow their own endogenously generated dynamics when spiking neurons release neuromodulators with each spike. Applying a rate reduction (detailed in Appendix section 1), we obtain the e-nmRNN rate dynamics, where á and 0 are nonlinear activation functions (e.g., e tanh and (<aeL.(taah))) for the and neuromodulator dynamics, respectively:
P = Φ Σ Wers +
Σ 3
TiskryTa +
Eualh
Σ waah
n4 = Σ Rejry +
Σ Zuim
These equations link biophysical parameters to trainable RNN parameters W, T, R, Z, The Zui
Table 1: Parameter and State Variable Definitions for a e-nmRNN
Parameter
Definition
State
Definition
Wij
Synaptic connectivity (i.j).
Ti
Firing rate, neuron i. Concentration kth NM.
Tijk
Modulatory effect of kth NM on synapse (i,j). Release of neuromodulator by neuron i.
Tik
Ris
Zki
Effect of neuromodulator I on k.
ua
Input current neuron i from stimulus L.
matrix models effective interactions between neuromodulatory systems (e.g., release of one NM influ- encing another), not literal chemical reactions, and often converges during learning to approximately diagonal.
Importantly, we can make explicit connections between the notion of "neuron-type" diversity and the structure of these matrices.
The sign of the columns W encode Dale's law enforcing the pre-synaptic neurotransmit- be either excitatory inhibitory [30].
The pre-synaptic partners have also been shown to encode cell type 31

---

The rows of the T tensor indicate the sensitivity of the post-synaptic neuron to the presence of volume transmitted small molecule n2 (t).
The columns of the R matrix indicate how much of each small molecule a neuron's activity promotes the release of, with a highly selective column indicating specialization. For in- stance, a unit whose activity primarily drives one modulator could be functionally described doopamine-promoiing unit".
The matrix defines the amount of amall-molecule to small molecule coupling 7 and often found be approximately diagonal.
Within the e-mmRNN model class consider three variants. The first axis of variation is the definition of the decoder. On one hand, we consider the classic trainable decoder with linear readout matrix D, and on the other, consider neuromodulated decoder where we put the output decoder subject to the neuromodulatory state:
0,(1) = (isT((t) or 0:(t)
We call these two options the "fixed decoder" and the "modulated decoder" respectively.
The second primary axis of variation considered in the work is the presence absence of spatial embedding and Dale's law. Recent work has shown that spatial embedding can improve the conver- gence compositional tasks [32]. improve interpretability (33, and reduce catastrophic forgetting through the mathematical connection learning with preferential attachment (34) This acts as
strong, iinsspe regularizer, imposing sparsity and encouraging local processing motifs.
To define our spatially embedded networks, we take inspiration from Waxman random graph genera- tion. We begin by hyper-uniformly placing N neurons into a D dimensional space, A (here D=2) [35 Next, we sample each cell randomly as an excitatory or inhibitory neuron denoted by 4 E {0, 1). Combining these two terms we generate our network connectivity with:
Wi = Wo(-1)ţe -Freyl
where W. and are the characteristic magnitude and lengthscale respectively. way is trainable parameter whose update rule is simply propagated through the backward pass to find a power- propagation-like learning rule (34 which causes large synapses learn more quickly and maintains sparsity of neurons which are not connected.
2.2 Multiplicative gating of dynamics
As noted by Costacurta et al [15], the nmRNN and this work's ss close cousin, the e-nmRNN, are special forms of gated RNNs (see also (T7).. While an RNN is universal function approximator, carries very different inductive bias than the gated RNN (36). To build intuition for the ramifications rank-2 neural network being modulated across a Bifurcation (e.g. Hopf) via multiplicative gating and additive synaptic drive respectively (see also work from Pellegrino et al 24 on a similarly formulated pitchfork bifurcation).
of this form of gating on sequence generation, we study the distinguishable signatures
We find that for a tanh() nonlinearity, these two mechanisms require very different symmetry properties and modes of action (e.g. gain modulation versus operating point shift). Further for broad class of symmetric nonlinearities, additive synaptic drive from a context neuron is less expressive than differential gain modulation. For more in-depth discussion see Appendix section 5.
While this intuition is essential to understand this work, believe that the interested reader will find this discussion insightful as we study training across various forms of Hopf bifurcations in sections 3, 4, and 5.
2.3 Interpretation as Context-Factorized Hypernetwork
We define context factorization as the decomposition network's effective connectivity into fast- fast-narying neural activity and slow-varying modulatory state (the 'context'; see figure [A].

---

Mathematically, the effective synaptic weight from neuron j to i is factorized as:
M
W (1) = W
+
(1)
This allows the now-dimensional context vector ri(t) to dynamically reconfigure the high-dimensional effective connectivity of the network. This is context-factorizze hypernetwork [15]19] 19]. where the NM network generates parameters for the main RNN. The term Σj, Ti represents tertiary interactions. Connections to SLDS, attention, and other architectures are detailed in Appendix
Context- Tarrext-Targeted Online Credit Assignment
Context factorization enables online, targeted credit assignment, which is idcally local in time and information 12
37
3.1 Deriving Learning Rules
Considering gradient descent on a loss L, we derive learning rules by matching terms order-by-order 11 [2].
For Tiya, the leading order gradient uses the pre-activation input ("synaptic drive") (1) to neuron i's nonlinearity and the instantaneous error signal e, (1) ya va (4):
V Tejx C.
x
Te
<a (al (A,()))
COnLEXT gate
Vw E, the usual gradient
This 4-factor rule is gated by (t), ensuring updates are directed the relevant context (18) by the neuromodulator and the transpose of the decoder matrix. D. This rule is a remporally-localize approximation to the full BPTT/RTRL gradient(12). The recurrent state r(()) and, critically, the slow neuromodulatory state vī(t) act as eligibility traces, carrying the necessary temporal context from the past, making the local update effective tasks requiring credit assignment over intermediate timescales. Other parameter rules are detailed in the Appendix section 3.
3.2 Connection to Meta-Learning and Gradient Approximation
The c-mmRNN dynamics allow w (f) to change via ri(t) as:
wo -Tggiu(t) Tipa0
ERISTY REJTY + Ezam
Σ
Can Tiyxeu (t) learn to approximate V w., L(1)? To study this question: we conduct numerical experiments in simple meta-learning sequence learning task where networks are trained generate specific example of the sine family of curves when fed in the amplitude, Ad, the frequency, w, and the phase, Φ; (see Figure 2 A). Our results show that trained c-nmRNN dynamics align with gradient descent on a surrogate network (Figure 2c, D). This aligmment (highly statistically significant, see Appendix section 4) implies ri()) can serve as an eтrоr readout, consistent with biological third factor 11]. This capability enables rapid in-context meta-learning [00 26).
Compositional Generalization via Sequence Mapping
Compositional generalization is crucial for flexible computation. We use "dictionary mapping" task: input [(() and output (1) are linear combinations of basis functions with shared latent coefficients ag (Figure 3A):
K
T(1)
QEPR((), (1),
y (1(
=
k=1
#=1
The network learn (o()()) R y (L), inferring scaling experiments, we note that task complexity scales with K. In keeping with the theme of oscillatory signals present broadly in the work, we study basis set of multi-period oscillatory signals.
reconstruct y² (1). As critical knob to conduct
5

---

Metalearning across family of with
Vanilla RNN
Gated Recurrent Unit
A
each sequence drawn from a distribution, y*))) Asin(ut + o)
B
6-nmRNN
6-nmRNN, spatially embedded
Max 11 A
I(t)
Loss
=Q
output Mett
di
~ 21[
- 0.1.
Model
Training steps
5000
C
Tyalk - -oV
P(t)
0.55
Similarity dynamics gradient
(2) V [T
Loss "epiphany"
Signn form
Statistical significance
r(t)
Surrogate
Waurrogate
Wi: + (t)
Training steps
Figure 2: Online and targeted credit assignment arises naturally from context factorization. (A) We define simple meta-learning task setup where input-output sequence pairs are generated from family of trajectories (here sine). B) c-nmRNN variants converge rapidly even compared to HP optimized (x100 iterations of TPE) baselines. Spatially embedded networks exploit both power- propagation
3 and context factorization to learn rapidly. C) To measure the alignment of the neuromodulator dynamics of the synaptie weights and the updates that follow the gradient, develop surrogate network comparison setup by syncing the nmRNN to an effective RNN. D) Across training, we observe learned alignment between the neuromodulator dynamics and the surrogate model's gradient. After the network undergoes an 'epiphany' in its learning, we measure statistically significant alignment. This suggests that the nmRNN has learned approximate gradient descent on the parameter updates within its closed loop dynamics (there are no gradient steps executed on the parameters within these validation trials).
We trained e-nmRNNs and baselines (Transformers, HiPPO-LegT (22), Oscillatory RNNs (38). GRU, Vanilla RNN) with matched parameter-count and hyperparameter optimization method (TPE sampling (39 (Figure 3C). e-nmRNN variants show strong compositional generalization across K, outperforming vanilla RNNs and matching GRUs (Figure 3D. E). However, the rich multiplicative dynamics of the e-nmRNN can also make it more prone to overfitting on smaller datasets compared to GRU, explaining its performance degradation at high task complexity. Spatially embedded (and Dale law enforeing) e-nmRNNs perform robustly overcoming common training challenges (37). The high coefficient decodability suggests the ability to learn disentangled representations mappable to ground truth that aid in generalization to never seen before combinations. We study this by uniformly sampling the hypercube defined by a space and set aside "quadrants" of combinations of bases for test set that are not seen during training. is noteworthy that the nmRNN did suffer from overlitting on small datasets with high complexity limiting its ability to compete with Attention and State-space models (see Appendix section 9.2). Future work will seek biophysical inspiration to directly address this pathology of the model class.
5 RPE Encoding and Learning in Reinforcement Learning
The capacity for online, in-context adaptation is particularly critical in dynamic environments requiring reinforcement learning [26). Previous experimental neuroscience studies have found neural correlates of crucial decision variables in reinforcement learning algorithms. For instance, dopamine activities are found to correlate with reward prediction errors [40] and prefrontal cortex neural activities are found to correlate with action values [41). We explored the e-nmRNN's capabilities of adaptation in such dynamic environments and examined the emerged neural representations using a dynamic foraging task inspired by neuroscience literature. In block-switched two-armed bandit task, where reward probabilities invert stochastically (Figure ) an agent must learn to track the hidden block state (i.c. which arm is the high probability) and adapt its actions to maximize reward.

---

Meta-Learning Compositional Generalization on Seq2Seq
[ Input ((t)
A
Input Basls k_k(t)
B
XK
TTask Compiexity
Transformers HiPPO-LegT
[Bass Generator (seed)
RNNN Oscillatory RNNs
(Fixed Task)
.+>> Output Basts k_k(t)
Vanilla RNN Gated Recurrent Unit c-nmRNNN, mod readout e-nmRNN, fixed readout e-nmRNNN spatial readout
[oefficient Generator - Coefficients a.k] -. Weighted Combination]
--> [Input [(0]
(Resampled for each batch)
(shared) +-..>> Weighted Combination ---> Target y"()]
[ Output y(!)
All models are Perameter Count Matched at -50k Compositional Generalization,
E
on
Explained (Val)
HP opt @
Decodabiliy
Structured Parzen Sampling Hyperparameter optimizatio
all
ectrect epochs explained
validation dataset
Performance Scaling "Task complesity, [1,2__00]
10
20
10
20
K, # components
K, of components
Figure 3: Meta- Meta-learning compositional generalization reveals the power of context factorization. A) Compositional task structure. B) Models learn to recognize basis functions, infer coefficients, and generalize to new combinations. C) Training setup with parameter matching and HP optimization is carried out across 4 steps. D) Performance scaling with task complexity (K) reveals nmRNN performance exceeding and on par with strong baselines. The model's higher capacity, however, can lead to overfitting on smaller datasets. E) Coefficient decodability with a linear probe shows robust representation encoding in simple environments with gradual loss task complexity grows.
Training e-nmRNN agents with Proximal Policy Optimization 42 across vectorized environments of this dynamic foraging task reveals the emergence of adaptive behaviors, including rapid adjustment of choices when the task context changes and anticipatory switching reflecting an internal model of the periodic environment (Figure 4B). Crucially, we find that the neuromodulator dynamics learn to robustly encode the task context (i.e. which arm has high probability) (Figure 4C, D).
Further analysis shows that the neuromodulator state also learns to encode the Reward Prediction Error (RPE) signal derived from a fitted Q-learning model (Figure 4 G). This is key finding: it demonstrates how the e-nmRNN can leverage its neuromodulatory system represent critical learning signal. Combined with the 4-factor learning rule (Section 3. I), where learning updates are gated by neuromodulator concentrations, this provides concrete mechanism for RPF-gated online credit assignment. This allows the network to dynamically adjust its effective connectivity based on surprising reward (RPE), facilitating rapid adaptation to changes the environment and contributing to the model's ability to "learn how to learn" in reinforcement learning context. Detailed experimental setup and analysis are provided the Appendix section 6.
Emergent Biological Signatures and Multitasking
This work echoes the promise of a bold hypothesis for grappling with neuro-biologien complexity: optimization biophysically conforming parameterizations across tasks which prioritize adaptability can provide principled lens into the paralyzing complexity of the brain (37 29 43) In humble support this perspective, we report that training spatially embedded, e-nmRNNs (Figure on the neuroscience-inspies multitasking benchmark from Yang et al. (2019) (27). which covers
wide array of decision-making cognitive tasks (see description in appendix 7). reveals emergent properties consistent with coarse- grained, neuro-bine ciooo-booooicall phenomena. While one-hot task ID is provided at the start of each trial, the network learns to maintain this information dynamically within its neuromodulatory state. The emergent properties include (Figure
Modularity: Emergence of functionally specialized modules and hierarchical cell clustering (see Figure 5D). See also complementary findings in Yang et al (2019) (27 and Driscoll et
7

---

A ((n)))
Actions
Misa
Right
Two-arm bandit with switching probabilities
NMs
Trials
D
E
1) [Tain Deep RL e-nmRNN on blocked bandit)
2) Fiit MIEE generated Action, Reward Sequences]
PCs
3) [Extract RPE from Q-table]
G
Neurona
1.
I DC
Trials
4) [Train (RPE, Choice, Reward, Entropy) decoder]
Neuromodulatorr
Figure 4: Reward Prediction Error (RPE) encoding emerges from RL training the e-nmRNN. A) Block-switcher two-armed bandit task. B) Trained agent actions show adaptive and anticipatory switching, C) Neuromodulator concentration dynamics learn encode context. D) NM state in PCA space reveals a cyclic trajectory linked to context. E) Example RPE trace from MLE fitted Q-learner. F) Single neuron encoding (via correlation) of behavioral features and RPE. G) Differential NM encoding (via correlation) of behavioral features, including RPE reveals learned symmetry breaking in NM roles in the dynamics.
al (2024) 28 Experimentally, this modularity supported in two ways: 1) the formation of pre-synaptic partners which both form "Sherringtonian circuits" [31 44 and 2) in modular brain regions (45). Furthermore, analysis of the trained network reveals that units specializing in releasing specific neuromodulator form spatial clusters, mirroring the organization of cell types the brain (see analysis in Appendix 8.3).
Cell-Type Specialization: Units specialize in releasing specific NMs 10 (Figure 5) Tijk defines cell types by their sensitivity to specific small molecules. Composed together, we may define the notion of a "dopamine-releasing neuron" as unit which dominantly promotes the release of single small molecule via volume transmission (i.e. approximating one-hot columns of the R matrix).
Timescale Separation: NM dynamics learn evolve on distinct timescales (e.g., 8x factor separating the fastest and the slowest) [10 (Figure 5T).
Context Encoding: NM state ii(t) learns to represent task context (Figure 5C). A linear decoder trained on NM concentrations reliably identify the task long after the initial cue has passed, whereas a decoder trained on neural activity cannot. This is a testable prediction for emerging, lemporally-resolver neuromodulator imaging technologies.
Through carefuul neuroscience inspired reverse engineering of trained networks, these findings support the biological plausibility and interpretability of the e-nmRNN framework. To contextualize these results, we trained a parameter-me matched standard RNN on the same benchmark. The spatially- embedded e-nmRNN achieves comparable or superior performance across the 20 tasks, demonstrating that inductive biases provide tangible value by yielding these emergent, interpretable structures that standard RNN does not as readily admit. Beyond this support for this class of nmRNNs, we view these results as small contribution this greater intellectual landscape attempting to reconcile the lessons of continual optimization in dynamic environment with the peculiarities of the brain.

---

Spatially embedded e-nmRNNs
Trained across many tasks.
NM learn to encode context
A
Portomanca
Malthen DM
No
Decodaall
14
Marden City
DR12
Wg (- W e
C14
DNM
0, it is excitatory
I, is inhibitory
Trais (1c6)
D
E
Neuromodulator dynamics learn to span nearly and order of magnitude.
Units learn release single neuromodulator
Modular Recurrent Networks with
Dale's law Generates Unit diversity
Neuromotuatto
8x
Postsynaptic
reterred
Auppomrdttoon
Presynaptic
Cell clustering
Sinilarity raloase
Amount
Time lags
release
Coupling
fring
Figure 5: Multitask learning with spatially embedded nmRNNs generates solutions mirroring neural data. (A) Spatial embedding neurons connect most strongly over short distances enforcing sparsity in the trained connectome. (B) Spatially embedded nmRNNs can achieve high performance across all 20 tasks. (C) Training linear decoder and reporting back the performance on a held-out validation trials, we find that NM concentrations encode context well after trial start, whereas rate encoding of context falls off quickly. (D) Emergent modularity and cell clustering suggests the formation of local circuits within E, Σx TiyRk. (E) The resulting R matrix designating small molecule release of the jth neuron reveals unit specialization
NM release (i.e. many neurons release a single small molecule dominantly). (F) Using an autocorrelation metric and its connection of Fluctuation- Duuctuation-iisiipatinn Dissipation theorem, we find that timescale separation NM dynamics spans 8x between the fastest and the slowest. This heterogeneity reflects the value of functional specialization.
7 Discussion
We interpret neuromodulated RNNs through context factorization via volume transmission, unifying NM roles in dynamics and plasticity. The e-nmRNN, derived from a biophysical model, offers a novel context-factorized hypernetwork architecture. This enables targeted online credit assignment and compositional generalization, supported by empirical results. Emergent biological structures in multitasking networks enhance interpretability. The framework extends to RL, where NM dynamics encode context and RPE, facilitating RPE-gated online adaptation. As recurrent architecture, the e-nmRNN inherently sequential during inference and less parallelizable than models like Transformers. This represents trade-off: we gain powerful inductive biases for online, continuous adaptation the cost of the massive parallelizability that makes Transformers scalable for large- offline processing.
7.1 Links between Neuro and AI
This work has significant ramifications for both neurobiology and artificial intelligence. For neu- robiology, the c-nmRNN provides computational hypothesis for the role of volume transmission and neuromodulation flexible behavior. suggests that context factorization is core principle implemented by these mechanisms to manage complexity and enable rapid adaptation. The emergent properties observed in trained e-nmRNNs, such as cell-type specialization based on release profiles and the encoding of context and RPE in NM dynamics, offer experimentally testable predictions about organization and function of neuromodulatory systems and computation with diverse neuronal components [2]. The distinct influence of multiplicative NM gating on network dynamics compared to additive inputs (see appendix section 5) also provides a specific computational signature that could be sought in biological recordings. In particular, the Hopf bifurcation for rank-2 network demands

---

differential modulation of the two modes by single neuro-modulator (i.e. one of the modes needs to be more or oppositely sensitive relative to the other).
Conversely, insights from neurobiology, particularly the architecture and function of neuromodulatory systems, provide a powerful source of inspiration for advancing machine learning. The e-nmRNN demonstrates how biologically inspired mechanisms like volume transmission and context factoriza- tion can lead more performant and, crucially, more interpretable AI models. The context-factorized structure offers a form of dynamic, interpretable control over network behavior, where the influence of different "context" signals (represented by NMs) on information processing can be directly analyzed. The emergent modularity and timescale separation suggest principles for designing more efficient and scalable AI architectures capable of handling complex, malti-tasking environments. By incorporating biologically plausible inductive biases, we can develop models with enhanced capabilities for online learning, meta-learning. and compositional generalization, addressing key challenges in current AI.
The concept of "computation with diverse components" is central to both fields. In neurobiology, different cell types and neuromodulators contribute distinct computational roles. In AI, designing systems with specialized, interacting modules is a goal. The e-nmRNN framework explicitly models this by incorporating different neuromodulator types with distinct dynamics and effects, and by showing how units can specialize in releasing specific modulators. This provides a concrete model for exploring how diversity components contributes to overall system function and adaptability.
7.2 Limitations
Our work has two primary limitations. First, the model's rich, multiplicative dynamics make it more prone to data-limited overfitting than simpler RNNs when training data is scarce. Future work will explore biologically-insiree regularization methods to mitigate this. Second, the neuromodulator dynamics have intrinsic, learned timescales; if task required extremely rapid context switching much faster than these timescales, the modulatory state would be unable to keep pace, leading to performance degradation. This highlights a fundamental trade-off between maintaining a stable context representation and achieving rapid adaptability. Finally, the hyperparameter search was extensive but non-exhaustive.
7.3 Broader Impacts
Our work contributes to understanding neural computation by bridging biology and AI. The model's interpretability aids in reverse engineering learned solutions and generating biological hypotheses, offering positive scientific impact with low misuse risk (see checklist).
We envision future value in pursuing the monumental task of translating directly between large comprehensive data sets such as the Allen Brain Cell Atlas [46) and compressing them through thoughtful reinterpretation into analogous W, R and T structures which can encode powerful initial conditions encoding strong evolutionary inductive biases for future application of the e-nmRNNs and similar model classes. These efforts promise insightful discussion within NeuroAI.
8 Conclusion
By formalizing volume transmission's 's role in context factorization, we introduce the e-mmRNN and join the growing chorus [16] 47 15 13 17 advocating for the unification of NM influence on dynamics and their role in plasticity gating. This framework enables targeted online credit assignment and robust compositional generalization, yielding emergent structures consistent with the rich complexity of neurobiology. The ability of the model's neuromodulator dynamics to encode critical learning signals like RPE highlights its potential for understanding and implementing "learning how to learn" in biological and artificial systems. This work underscores the power of neuro-AI, where insights from neurobiology inspire novel, interpretable, and performant machine learning architectures, while computational models provide framework for understanding complex biological phenomena like computation with diverse neuromodulatory components. We view e-nmRNNs as a promising model class that bridges AT and neuroscience, highlighting opportunities to advance our understanding in both domains.
We make our code available
10

---

Acknowledgements
We gratefully acknowledge funding and community support from the Shanahan Foundation Fel- lowship (MSB and ALS) and the University of Washington's 's Computational Neuroscience Center (MSB, PCK, and ALS). We acknowledge compute resources provided by the Allen Institute for Neural Dynamics and the eSciences Institute at the University of Washington. We thank Ulises Pereira-Obilinovic, Matthew Golub, Jacob Sacks, Ryan Raut, Anamika Agarwal, Harsha Gurnani, Uygar Sumbul, Bing Brunton, Eric Shea-Brown, Adrienne Fairhall, Laura Driscoll, Kyle Aitken, Stefan Mihalas, Marton Rosza, Kayvon Daie, Karel Svoboda, Christof Koch, and the entire Allen Institute and University of Washington ecosystem for insightful discussions and encouragement. We also thank the Allen Institute founder, Paul G. Allen, for his vision, encouragement, and support.
References
[1] Eve Marder, Sonal Kedia, and Ekaterina Morozova. "New insights from small rhythmic circuits". In: Current Opinion in Neurobiology 76 (Oct. 2022), 102610. ISSN: 0959-4388.
DOI: 10 1016 conb. 2022. 102610 URL: https : // www sciencedirect. com (visited 05/06/2025).
[2] Cornelaa I. Bargmann. "Beyond the connectome: How neuromodula-
tors shape neural circuits". en. In: BioEssays 34.6 (2012).
_eprint:
http://.ooommooooo////oo/0/////0 1002/bies.201 10111085, pp. 458-465. ISSN:
1521-1878. DOI: 10 101100000///// http://beeee.................... 201100185
URL:: https:::... : owwionniniaarryyyy wiley
com/àoi/abs/10 10w2/00ee..ee.e 201100185 (visited on 05/06/2025).
[3] Yuhan Helena Liu al. "Cell-type-sppe specific neuromodulation guides synaptic credit assignment
in spiking neural network". In: Proceedings of the National Academy of Sciences 118.51
(Dec. 2021). Publisher: Proceedings of the National Academy of Sciences, e211 1111881111 18.. Dor:
10. 1073/ 1 pnas 11118211111 URT.: https: http:::/wwww //www//wwwwww pnas. wwwppppa......ppp..pppppp http/////ffll//1///// 1073/ pnas
2111821118 (visited on 05/06/2025).
[4] Yuhan Helena Liu et al. "Biologically-plauiibb backpropagation through arbitrary timespans via local ceuromodulators" en. In: Advances in Neural Information Processing Systems 35
(Dec. 2022). pp. 17528-175444 17542, URL:: htttps://proceed ieerreceerdnnnnnnnnneeeee..eeep neurips neurips.cc/paper. files/ paper/2022 hash afccaeeedccaaffeeeeeaaaaaaaaaaae -ttp:/aasataattaeaaaaaaaaaaaataa html (visited on 05/06/2025).
Abstract Conference
[5] Özge D. Özęete. Aditi Banerjee, and Pascal S. Kaeser. "Mechanisms of neuromodulatory
volume transmission".. en. Molecular Psychiatry 29.11 (Nov. 2024). Publisher: Nature Publishing Group, pp. 3680-3693. ISSN: 1476-5578, DOL: 10. 1038/ -00000022----00
3 URL: https: ///ww.. 05/06/2025).
www. nature. com / /ar/ariiclee s41380 024 02608 3 (visited on
[6] Jinyung Hong and Theodore P. Pavlic. Learning Modulate Random Weights: Veuromodtulation-iispiiee Neural Networks For Efficient Continual Learning.
arXiv:2204.04297 [cs]. Oct. 2023. DOI: 10 48550 arXiv 2204 04297
URL:
http://arxiv.. org/ abs//204..0eeeeeeo
(visited on 05/06/2025).
[7] Allison Cruikshank et al. "Dynamical questions in volume transmission". In: Journal of Biological Dynamics 17.1 (Dec. 2023). ISSN: 1751-3758. DOI: 10. 1080/17513758 2023
2269986 (Visited on 05/08/2025).
Rachel Grashow, Ted Brookings, and Eve Marder. "Reliable neuromodulation from circuits with variable underlying structure". In: Proceedings of the National Academy of Sciences 106.28 (July 2009). Publisher: Proceedings of the National Academy of Sciences, pp. 11742- 11746. DOI: 10. 1073/ 090001905510 11166 URL: Attpps:/wwxs http//wwwww.....//////////. wttp:pass....o.i.pippp
pnas. 0ww.00055111100 1066 (visited on 05/06/2025).
[9]
Michael C. Avery and Jeffrey I.. Krichmar. "Neuromodulatory Systems and Their Interac-
tions: A Review of Models, Theories, and Experiments". English. In: Frontiers in Neural Circuits 11 (Dec. 2017). Publisher: Frontiers. ISsN: 1662-5110)
DOI: 10 3389 / fncir 2017. 00108 URL: hhttps::/www..c http://wwwfoottrriinnn. orghttps: //www frontiersin org/ Bournals/neurall hoo.raa/i//eeaaacccccittttttttta 3389 fncir. 2017 00108/00111/F10 full (visited on 05/06/2025).

---

[10] David A. McCormick, Dennis B. Nestvogel, and Biyu J. He. "Neuromodulation of Brain State and Behavior". en. In: Anmual Review of Neuroscience 43. Volume 43, 2020 (July 2020). Publisher: Annual Reviews, pp. 391-415
ISSN: 0147-006X, 1545-4126. Dor: 10 1146/
annurey-neuro- 1-w-11000111---- journals/10.. 100 1 annurev newroo 100299 (viis
URL.: https: // annualreviews wttp:viwwwwwwwwooeeeeeeeeee (visited on 05/06/2025).
[11] Guillaume Bellec al. "A solution to the learning dilemma for recurrent networks of spiking
neurons". en. In: Nature Communications 11.1 (July 2020). Publisher: Nature Publishing
Group, p. 3625. ISSN: 2041-1723. Dor 10 1038/ s11447 020 17236- y! URL.: https
7/www. //www.nature.a. conttuurecomaaaaa www.ao/ttweee///o........oootaet sawee////444777200000 17236-2226111 (visited on 10/21/2024).
[12] James M Murray. "Local online learning in recurrent networks with random feedback". In: eLife 8 (May 2019). Ed. by Peter Latham, Michael Frank, and Brian DePasquale. Publisher:
eLife Sciences Publications, Ltd, e43299. ISSN: 2050-084X. DOI: 10. 700.7555//e.l www.ee..eeeeee.eeeeeeeee eLiff.43299 7554ee117555f 43299 (visited on 10/21/2024).
URL:: httt::ttts:///d/e
[13] Thomas Miconi et al. "Backpropamine: training self-modifying neural networks with differ- entiable neuromodulated plasticity". In: Sept. 2018. URL: http://ttss/oooeerrreie net/
forum?id=r111A41551 (visited on 05/06/2025).
[14] Sarthak Mittal et al. "Explicit Knowledge Factorization Meets In-Context Learning: What Do We Gain?" en. In: May 2024. URL: hatt:://ppeeerriieeeeeee/ooorrrr (visited on 05/08/2025).
[15] Julia C. Costacurta al. flexibiiityyy in recurrent neural networks via necomoaaaoo..
en. Pages: 2024.07 0..4.07.22 26.65655 Section: New Results. July 2024. DOL: 10. 1ww./11210011100 26
605315 URL: https: :/wp::://wwww.. htt///wwww.oo..i.o/////// biorxiv. www.xwwwwwg/oooeeeeeeeeeeeeeeeee wttp://oootttt/...11100022222220 (visited on 10/21/2024).
110.....112222111 07. 605315vv 26.6665555
[16] Kyle Aitken and Stefan Mihalas. "Neural population dynamics of computing with synaptic modulations".. In: eLife 12 (Feb. 2023). Ed. by Gianluigi Mongillo, Joshua Gold, and Omri Barak. Publisher: eLife Sciences Publications. Ltd. e83035. ISSN: 2050-084X. DO1: 10. 7554/ eLife 83035 URL::httts:///at
org/ 71/1//10.555//// wwwpe//eiffe..e. 83035 (visited on 05/06/2025).
[17] Ben Tsuda et al. Neuromodullators generate multiple context-relevant behaviors in a recurrent neural network by shiftting activity hypertubes. Pages: 2021 .0..5 05.31 446462 Section: New
Results. Jan. 2022. DO1: 10. 1101/2021 05 31 446462
URL: https: h/www/bborxxxivv 1010/1122222 05. 31. 446462v2 (visited on 05/15/2025..
org/ ctt//onnttttttceecc
[18] David Ha, Andrew Dai, and Quoe Le. HyperNerworkss wwtp://www...o.oiio..000000...0.
Dec. 2016. DOI:
1609. 10. 00.48550/arx11 att.://rxii....... 1609 www.6.99..011.88 09106 T0/21/2024).
URL:
(visited on
[19] Johannes von Oswald et al. Continual learning with hopphneewworks.... arXiv http:://wwo...aaiix...9000000000 Apr. 2022. DOL: 10. 48544/50//aa 118500/arXiv.. www....vw........... (visited on 10/21/2024).
URL: htL::htt:://// httpaaawiv.orggg...// httpaaab///.....
[20] Johannes von Oswald et al. Transformers learn in-context by gradient descent.
arXiv:2212.07677. May 2023. DOI: 10. 18550/arXix0 22122.0777 URL: http:://arx:
org/ 2rg/abs/2212.0r1 07677 (visited on 10/21/2024).
[21] Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber. Linear Transformers Are Secretly Fast Weight Programmers. arXiv:2102. 1174. June 2021. DOI: 10. 48550/ 38650/arXiv..o arxiv. 2102. 11174 URL: hbtp://aarriv.orr//bb/22222 11174 (visited on 10/21/2024).
[22] Albert Gu et al. "HiPPO: Recurrent Memory with Optimal Polynomial Projections". In: Ad-
vances in Neural Information Processing Systems. Vol. 33. Curran Associates, Inc., 2020, pp. 1474-1487 URL: https proceedings neurips cc paper 2020 / hash /
102f0bb6ef b3a.0666e33366100 2aa.11a2333c3cc0 htt.:/aa622222777000111666066600 Abstract html (visited on 05/06/2025).
[23] Benjamin Ehret et al. Continual Learning in Recurrent Neural Nenworks. arXiv:2006.11109
Mar. 2021. DOI: 10. 485404550/000 art/v..X.v... 2006 12109
URL:: http: http:/iiiv.rrr///////////
12109 (visited on 10/21/2024).
[24] Arthur Pellegrino, N. Alex Cayco Gajic, and Angus Chadwick. "Low Tensor Rank Learning of Neural Dynamics". en. In: Nov. 2023. URL: httpss://pperrevieer net/forum forum ?
WcoX8eJJjI (visited on 05/08/2025).
[25] Marcin Andrychowicz al. Learning to learn by gradient descent by gradient descent, arXiv:1606.04474 Nov. 2016. DOI: 10. 00.44855//r/// 1606. 04474 URL: http://arxivv
org/abs/1606. 04474 (visited on 10/21/2024).
12

---

[26] Jane X. Wang et al. Learning to reinforcement wtw.::mm..0i0111111111111 [es]. Jan. 2017. DOI:
10. 41.44855//a0 araia. 111..1111.1000 05763 05/15/2025).
URL: hRt: httpt::/paaii...ttrt/// org/ abaaaaaa/b111 05763 (visited on
[27] Guangyu Robert Yang al. "Task representations in neural networks trained to perform many cognitive tasks". eng. In: Nature Neuroscience 22.2 (Feb. 2019), pp. 297-306. ISSN; 1546-1726. DOL: 10. 10011003311550 0018//41593--111 018 018-0000-2
(28) Laura N. Driscoll, Krishna Shenoy, and David Sussillo. "Flexible multitask computation in recurrent networks utilizes shared dynamical motifs". en. In: Nature Newroscience 27.7 (July 2024). Publisher: Nature Publishing Group, pp. 1349-1363, ISSN: 1546-1726. DOI:
10. 10.1038/s478 011113//41118-02----1111-0
URL: https://ww/t nature com httpe///cc//aaiieeee///11/00////
024-01668-6 (visited on 10/21/2024).
[29] Anthony Zador "Catalyzing next-generation Artificial Intelligence through NeuroAI". en.
In: Nature Communications 14.1 (Mar. 2023). Publisher: Nature Publishing Group, 1597. ISSN: 2041-1723. DOI: 10.
023 37180- URL: https: ///ww// nature
com/articles://44 - 023- 37180- (visited on 05/15/2025).
[30] Jonathan Cornford et al. "Learning to live with Dale's principle: ANNs with separate excitatory
and inhibitory units". en. In: Oct. 2020.
URL: https: //tp:///opppriiieeeee aet/foruu ? idd
eU7762YxEpz (visited 05/15/2025)
[31] Casey M. Schneider-Mizell et al. "Inhibitory specificity from connectomic census of mouse visual cortex". en. In: Nature 640,8058 (Apr. 2025). Publisher: Nature Publishing Group,
pp. 448-458. ISSN: 1476-4687.
DOE: 10 1038 / s41586 024 07780 URL: https //www.natur com/ hamm/articlessssee 541566--1188118/ 0011---22--777-7--7-7 (visited on 05/15/2025).
[32] Mikail Khona et al. "Winning the Lottery With Neural Connectivity Constraints: Faster Learning Across Cognitive Tasks With Spatially Constrained Sparse RNNs" In: Neural
Computation 35. 11 (Oct. 2023). pp. 1850-1869. ISSN: 0899-7667. DOI: 10. 11622/ecooo__
01613 URL: https:://///. //doi. org//00//0/0.. www.:1162/ncccoeecc _ _atteeeoo____1_133e1___11111113e (visited on 05/06/2025).
[33] Jascha Achterberg et al. "Spatially embedded recurrent neural networks reveal widespread links between structural and functional neuroscience findings". en. In: Nature Machine Intelligence
5.12 (Dec. 2023). Publisher: Nature Publishing Group, pp. 1369-1389 1381. ISSN: 2522-5839, DOI:
10. 10.1088/442255 023 - URL: https: sttps::/www// http://ww.at....eeeeeeeeeeee.eee wwwurantteeeeee com wwwarriiieeseeeeeeeeeee
023-00748-99 (vii0 on 05/15/2025).
[34] Jonathan Schwarz et al. "Powerpropagation: A sparsity inducing weight reparameterissttion en. In: Nov. 2021. URL: httpps://ppenreviieeee net net httt/forummiddags1luaa 05/15/2025).
(visited on
[35] Salvatore Torquato. "Hyperuniform States of Matter", In: Physics Reporty 745 (June 2018). arXiv:1801 100.080006624 [cond-mat), pp. 1-95. ISSN: 03701573, DOI: 10. 001:|10.11 1016/j phwpsrpp.ppppp.... 2018
03.001
URL: http:///axiz: http://axxiv.oogabbb/////ss 1801/11111111 06924 (visited on 05/15/2025..
[36] Kamesh Krishnamurthy. Tankut Can, and David J. Schwab. "Theory of Gating in Recurrent Neural Networks". In: Physical Review X 12.1 (Jan. 2022). Publisher: American Physical
Society, p. 011011 DOL: 10. 1103 1103/PhysRevX. 12. 011011 URL: https://link aps. doi/10. 1w111100//hhhheeee 12. 011011 (visited on 05/16/2025).
[37] Blake A. Richards et al. "A deep learning framework for neuroscience". en. In: Nature Neuroscience 22. (Nov. 2019). Publisher: Nature Publishing Group. pp. 1761-1770. ISSN: 1546-1726. DO1: 10. 1038 s41593 019 0520 2 URL: Ihttps: //t/w/ww nature. com/
articles/s41593 011-599--199-000 0520 (visited on 05/08/2025).
[38] T. Konstantin Rusch and Siddhartha Mishra. "Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies". (visited on
en. In: Oct. 2020. URL: httttttt/p///erreeett www.eewwweeeeeeeeeeeeeeeeeeeeeee /tte:/orrrmn---tasaeaiii 05/09/2025).
[39] Shuhei Watanabe. StretStructured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance. arXiv:2304. [es]. May 2023. DOI: 10. 48550/555/// arxiv. 2304. 11127 05/09/2025).
URL:: http: htap::///axiivi org/ 2233032//////...////////// 11127 (visited on
[40] Wolfram Schultz. "Predictive reward signal of dopamine neurons". In: Journal of neurophysi- ology 80.1 (1998). pp. 1-27.
[41] Bilal Bari et al. "Stable representations of decision variables for flexible behavior", In:
Neuron 103.5 (2019), pp. 922-933.
13

---

[42] John Schulman et al. Proximal Policy Optimization Algorithms. wrt.: [cs]. Aug. 2017. DOI: 10. (visited on 05/16/2025).
48550/555KK... 20155//rxxv... 1707 06347 http::/aa// wwt.:o/////aaaaeee 06347
[43] Jane X. Wang al. "Prefrontal cortex meta-reinforcemeen learning system". en. In: Nature Neuroscience 21.6 (June 2018). Publisher: Nature Publishing Group, pp. 860-868.
ISSN: 1546-1726. DOL: 10. 1038/
0147-8 URL: stttps//viin www.:naaur.... com/
#tticceess41153--08---0-11
(visited on 05/15/2025).
[44] David L. Barack and John W. Krakauer. "Two views on the cognitive brain".. en. In: Nature
Reviews Neuroscience 22.6 (June 2021). Publisher: Nature Publishing Group, pp. 359-3 371.
ISSN: 1471-0048, DOI: 10 1038 /0088//41500 s41583- 021 00448-0480000 URL: https: :/www.
www. nature
com/ 00mm/2rrtcceee//45588-20222--2-2 00448- (visited on 05/15/2025).
[45] David Meunier, Renaud Lambiotte, and Edward Bullmore. "Modular and Hierarchically
Modular Organization of Brain Networks". English. In: Frontiers in Neuroscience 4 (Dec.
/ 3389 fnins 2010 00200, URL: https / www hwtp:wfoootiioe frontiersin org/ journals /tw.//eeuurnnneeeeeeeeeeee hetpeeennncrrrttleeeeeee artiicattels.... 10 3389
2010). Publisher: Frontiers. ISSN: 1662-453X. DO1: 101:/10.0
fnins. 2010. 00200/ful1 (visited on 05/15/2025).
[46] Zizhen Yao et al. "A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brain". en. In: Nature 624.7991 (Dec. 2023). Publisher: Nature Publishing Group. pp. 317-332. ISSN: 1476-4687. DOL: 10 1038 / s41586 023 06812 URL: 023 06812-2 (visited on 05/16/2025).
https://wws natura. com/ www.aarrcceesesse1
[47] Thomas Miconi and Kenneth Kay. " An active neural mechanism for relational learning and
fast knowledge reassembly". eng. In: bioRxiv: The Preprint Server for Biology (Sept. 2023),
p. 2023.07.27.550077
ISSN: 2692-8205. DOT: 10. 1111111000000 20200/222..00 07 27 20..27.555777 550739
[48] Scott W. Linderman et al. Recurrent switching linear dynamical systems. Oct 26, 2016. DOI:
48.505 aar00/ariiv... 1610. 08466 arviv: 1610. 08466 [stat]
URL: http: org/
abs/1610. 08466 (visited on 10/24/2025).
[49]
Steven L. Brunton et al. Modern Koopman Theory for Dynamical Systems. ar11v:1002.2000 Oct. 2021. DOI: 10. 48550 aat/arXXiivv.... 2102. 12086
URL: http:://tt/at 2wtpax/iii...ra///////222/2
12086 (visited on 10/21/2024).
[50]
Michael T. Pearce et al. "Bilinear MLPs enable weight-based mechanistic interpretabilityy. en. In: Oct. 2024. URL: 05/08/2025).
net wntr://orru///ddddg0000t..00 (visited on
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We discuss analytically and experimentally how volume transmission can im- plement context-factorizztion and how this computational primitive enables targeted online credit assignment. This work views this implementation through the lens of targeted online credit assignment and learning compositional generalization in carefully designed meta- learning and multitasking experiments. Importantly, we highlight not just the performance of the model class but also its ready neurobiological interpretability.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We explicitly discuss limitations of model class overfitting and extensive but eeha-uxaustive hyperparameter searches for all experiments reports in the "Limitations" section of our discussion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
15

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All theoretical results are given extensive space the supplementary informa- tion documents and appendix including:
the full derivation of the rate reduction
the connection to Fast weights and by extension linear transformers
the derivation from Matrix Taylor expansion grounds the next leading order contribution to the RNN.
the connection to state space models
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the suppiemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We make explicit effort dedicate substantial space the methods and parameters of each experiment in the main text and provide further implementation tables in the appendix. [See below for code availability).
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
16

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will release the full experiments on github and associated docker instances hosted on CodeOcean for ready reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. wwwpppblccgguieeeeeeeeoeeeppeeee ppw.liicgguudss//oeeebbbiisssss
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: To the fullest extent possible, provide the workspace of each of experiments the appendix. Large fractions of the selected hyperparameters where chosen in the work using HP optimization in the form of TPE sampler (implemented in Optuna).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
17

---

Answer: [Yes]
Justification: To the extent that compute limitations allow on our extensive sweeps, we represent the variability of our results in the form of error bars reporting the standard deviation of exkerimxeeillttlll results. These are reported as error bars on the mean result. For example traces such as in figure 2D and Figure 5, we make every effort to choose representative samples from the distribution (ncither the best experiment we ran nor the worst).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: All compute resources are completed on the cloud within an AWS environment requiring less than 64 GB of RAM and CUDA enabled GPU. Each experiment's 's compute resources are provided in the appendix documents under "Experimental details".
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::/ http:://eeuris.eeee http:rpiss..ppplliehhtthctecceee
Answer: [Yes]
Justification: We have considered both the consequences of pursuing the research on environ- mental resources and believe that the work to make learned dynamics readily interpretable worth the relatively small costs. We have considered the ethical ramifications of inter- pretable models inspired by the brain and consider the distribution of uses as acceptable not beneficial to the larger community.
18

---

Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We include a discussion in the main text of our broader impacts. We welcome engagement from the community.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do introduce/release models with high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
19

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All licensed code is credited in the code base and through explicit citation in either the paper or the appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperssi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All assetts generated in the course of this work are in the form of code and manuscript and will be licensed under CC-BY 4.0 designation. We inluede structured templates in the appendix.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No human subjects used in this study.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
20

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No IRB approvals required for this study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
We provide description in the appendix. For reference, LLMs were not used for core methods, but did help to structure the logging backbone (with specific focus on the naming convention) in two of the experiments. These generated results were carefully checked for accuracy and all changes outside of this purpose to the code were reverted in using pre-post file diffs. One alternative visualization barplot representation not used the final form of the manuscript was generated from code provided by an LLM.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMMs any important, original, non-standard components.
Please refer to LLM policy for what should or should not be described.
21

---

Appendix
1 Detailed Derivation of e-nmRNN Dynamics
LIF Neuron Model with Neuromodulation via volume transmission
To capture neurobiological realism, we augment the Leaky Integrate-and-Fing (LIF) neuron model to include the influence of multiple neuromodulator types. The derivation begins with the assumption of a "well-mixed" neuromodulatory environment. This is principled abstraction representing timescale separation, where diffuse neuromodulator concentrations establish a stable, slow-varying "context" for the fast neural dynamics (i.e., Tn > TP). This is biologically motivated by two observations: 1) The integrative nature of NM concentration acts as a low-pass filter on population activity, which in turn modifies the mean population dynamics, useful tool for creating multi- timescale dynamics. 2) The wide cortical projections of single NM-releasing neurons suggest biology may accelerate the approach well-mixed state by increasing release site density. We begin with network of LIF neurons. The subthreshold dynamics of the membrane potential Vi(t) for neuron i given by:
dV. = -v E1) + (t) + (ezt,i(t)
Cm di
where Cm is membrane capacitance, 9L is leak conductance, EL is leak reversal potential, (t) is the total synaptic current, and (t) ) is external input current. A spike is emitted when V2(() reaches threshold Via. followed by reset Vreset and refractory period Tref-
We introduce M types of neuromodulatons with concentrations (4),), k I,..., M.
Each cell will be defined by a cell type which controls the distribution of its parameters.
These parameters are M-dimensional vector (one number for each of the M neuromodulators) which control:
1. I, the ncuromodulator current output with spike
2. T, the timescale of the release from that single cell
3. Ig the cligibility current signaling the sensitivity to each neuromodulator
4. F. the decay timescale of the eligibility
We constrain these neuromodulator release dynamics such that that shorter acting neuromodulators have a correspondingly higher release. The idea is that we can normalize the area under the curve rather than have faster decaying NMs having significantly less potency.
These parameters control both how much each cell contributes to the pool of neuromodulators and how sensitive they are to that pool via multiplicative gating.
Every cell now has a larger state, the membrane potential plus the eligibility for each the neuro- modulatory molecules. There is also shared state for the whole network interacting through the well-mixed neuromodulatory pool.
Postsynaptic and Presynaptic Effects on Synaptic Current
Neuromodulators can affect neuronal excitability (postsynaptic) and neurotransmittee release (presy- naptic). We model these combined effects by modulating the effective synaptic strength. The total synaptic current (t) = Σ (t), where Ley(t) (t) is the current from presynaptic neuron j.
Let i/ be the time of the f-th spike from neuron j.
(t) = Σ alt - t/) - (t)
(2)
Here, att) is the postsynaptic current kernel (e.g.. a(t) the effective synaptic strength. We model the neuromodulatory effect on this strength as:
Quze e t/Teym for t > 0), and w
(t) is
Tayn
weft (t)
Wi
+
TLI 2jk
(t)
(3)
22

---

where : is the baseline weight and T quantifies the sensitivity of the synapse + i to modulator k. The superseript 'LIF' distinguishes this parameter from the final RNN parameter.
Substituting into the expression for (t):
N
M + Σ
alt - 12)) (1) = ΣΣ
W
TLIF ojk
mx(())
(4)
5 1
Neuromodulator Dynamics
The concentration (t(t) evolves based release, degradattio//uttake, and interactions. Let Sy(t)
E, 6(1 - ief) be the spike train of neuron j. We model the dynamics as:
dr Tn.k dt
M
-2k + Σ
Tnn BARRRIIS )st(t)
Σ Z ((((()))) (t(
(5)
1=1
Here, Tn. is the time constant for modulator k. R is the amount of modulator released per spike from neuron j. and Z represents linear interaction effects between volume-transmitted, small molecules, including decay modification (via Z (7). The interaction term Z models effective interactions, not literal chemical reactions. allows for one neuromodulatory system to influence another (e.g., release of one NM promoting or inhibiting the release/clearance of another). In practice, we often find this matrix is learned as approximately diagonal, simplifying self-decay terms.
Rate Reduction
We transition rate-based description by averaging over spike times, replacing detailed dynamics with average firing rates (t).
Approximating Firing Rate
The average firing rate rt( is approximated as a non-linear function or (the f-I curve) of the mean total input current ((1) (1)),,(()) + (1)),
ri(t) = (f((Ttot,:(t))
Defining and Training Excitability
Excitability at the single neuron level is encoded in the nonlinear function +() that translates the synaptic current into the resulting firing rate. The use of such nonlinearities standard in rate-based models to ensure stability and represent the saturating, positive firing rates of biological neurons. These nonlinearities also provide important numerical stability during training across much wider range of byperparameters.
One way conceptualize excitability is in terms of LIF neuron which modulates its excitability by way of changing the threshold needed to cause a spike, 7. The Leaky Integrate and Fire neuron can be written in simplified form:
0 = kv + I s(t)ureset
and is augmented by the fire part of the relationship with the nonlinear spiking function:
st) = {1
0 vtt - <T if v(l - T
(8)
To find the resulting steady-state firing rate of LIF neuron for a given driving current, I, we can solve this equation in piecewise linear fashion solving the the differential equation for the time reach threshold to find that the firing rate takes the form of:
1) -
KT
fLIF(1) = -k
(9)
kV <Vrrsse
23

---

4
Rate
1-
0
2
Current
Figure 6: A critical step in the transformation between the LIF neuron and the rate neuron is the shape of the nonlinear mapping between DC input current and output firing rate the limit of firing rate being fast compared to the dynamics of the input current. We note caution as this assumption only holds for slow directions in the dynamics space. Under this set of assumptions, we derive closed form equation for the firing rate by completing the piecewise integral of the time to achieve threshold and inverting that to define firing rate. Notice that this firing rate most closely follows logistic or the ReLU-Tanh nonlinearity in that it incorporates both critical threshold for firing and a saturation brought about by the refactory period. Colors in the plot indicate the changing of the threshold of the LIF with direct correspondence to the changes in the slope and the critical current needed to induce firing.
This equation allows us to map the dynamics of a single neuron into a steady state approximation of the firing rate as a function of the input current. This also allows to study the effect of changing the 'excitability' of the LIF neuron through change in its threshold to fire. This can be visualized following:
There are three noteworthy aspects to a change in excitability through change in the threshold.
1. With increasing excitability, the threshold from not firing firing decreases.
2. With increasing excitability, the slope of the characteristic linear regime increases
3. With increasing excitability, the neuron reaches its maximum firing rate at a smaller input current.
There are then two ways in which we can train the excitability. One, we can directly parameterize our nonlinearity as the LIF rate and use gradient descent to update the LIF threshold value T.
Φ() fLIF (I) -f=
Or second, parameterize the nonlinearity in terms of two coupled parameters that merge approaches in the literature for training neural excitability:
Φ () (eRee.(()anh)))) + +e;)
where the slope and threshold of the each neuron's nonlinearity is co-modulated by their gradients the loss function:
E, - Ve, L
(10)
è = Ve
(11)
The current implementation uses the second form in terms of the ReLU (tanh () nonlinearity.
Approximating Average Synaptic Input
We replace spike trains Sy(t) with rates (t). From 4 assuming ny and rt()) change slowly relative to ox()):
N
M
Uoym,i(t)
alt t/)1
Wi + Σ
LIF Tisk k=1
ma(t)
(12)
24

---

The average current from neuron j is at = ry(t)- Qior. where Qiot = 0000 a(t))dt. Let's define the RNN weights by absorbing Qtot: Wi Woor and Tiye QroeT
(1)) =
E rt((t
Wij +
Σ Tisama(t)
(13)
j=1
Wtrry(t)+ +
Σ
Tesher(t)mm(()
(14)
j= ==
Approximating External Input
Assume external input (t) arises from L signals [t(t) weighted by ugz
(Teat, (l))(
(15)
Rate Dynamies Equation
Standard rate models include time constant Tr for rate dynamics:
Tr dr:
(t) or ((Ieo,,
(16)
dt
Substituting [1p and 15
dr: Tr dt
N
N
M
(t) + or
Σ Worr(++ +
(1) + Σ uaTi())
(17)
Neuromodulator Dynamics in Rate Terms
Replace (y) with (t) in 5):
dnk Tn.k
Thk
Σ 5=1
(Tn R )r(()))(())))))))))))))) + Σ (Tr, Z )ni(1)
(18)
dl
Define RNN parameters R,j Tn,k RI.IF and effective interaction/decay term Zu such that the equation becomes (after swapping index l for the equation, and I - A: for the sum):
N
M
dng
Σ j=1
(t) + Σ (tn)(t)
(19)
dt
k=1
Note that Zi combines the original n1; term and the ZLIF terms. Explicitly, let 2 Tn Tn,i2LIF. LIFF
Then the right hand side is -n, + Σ, RiyTy + Σg Zg Grouping terms involving nig, we get
Σ, Rejsy + (Zi 6 my. So, 7 7i 8zk-
Matching the Target Functional Form
We now compare the derived rate equations
and 19 with the target RNN forms the main text:
f =
Σ WIsTy + Σ
To +
Σ uahi
(20)
j.k
n; = Pn Σ RISTy + Σ Zsana
(21)
25

---

Rate Equation (7)
Derived: T, ri + 0r (Input,))
Target: ñ; = (Inputt,)
where Input; = Ej Weyry Ds, TighTyne Equah.
The crucial outcome is that the argument of the non-linear function o, (or Φ) has the structure Wor, " + ual,, which directly reflects the baseline synaptic drive, the neuromodulated synaptic drive (via effective weight modulation W ), and external drive derived from the LIF model.
Obtaining the exact target form r; = @(Input,) requires interpreting it as a specific modeling choice where the rate dynamics might lack the explicit decay term (-r;) absorb it and the timescale T, into the definition of Φ. The key justification for the target form's structure comes from the LIF derivation of the input terms.
Neuromodulator Equation (n)
Derived: i = =, Rigsy Ex Zi (a linear system) Target: in; on (2, BiyTy + Ee Zamus (a non-linear system)
The derived dynamics for n, are driven by terms related to release CS Reyry) and interactions/decay <x n+). The target form posits a non-linear function on governing these dynamics, likely representing saturation effects or more complex regulation
captured in the simple linear model (5) The structure of the argument inside 2R namely Σ, RigTy + Ee Za.ny is consistent with the derived driving terms (where Zik in the target form corresponds conceptually to Z/μ from the derivation, potentially absorbing ;. Using the same functional form á as in the rate equation is further simplifying assumption of the target model.
Conclusion of Derivation
Starting from a LIF neuron model incorporating synaptic modulation (Woff Wij + En TLFE iik mg)
and activity-dependen neuromodulator release/dynamics (driven by PLFF LL we performed rate reduction. This yielded:
An average input current to neuron with terms corresponding to baseline synapses (WiyTz), modulated synapses (TijaTyma) and external inputs (uali).
A dynamic equation for neuromodulator n, driven by release (Reyry) and interactions/decay (z).
The target RNN equations 20 and 21 represent a specific phenomenological model choice for the rate and modulator dynamics (i = o(...),
o.....)). The essential link provided by this derivation is the justification for the structure of the arguments within the non-linear functions ø and Φn. These arguments directly map to the effective inputs and driving forces calculated from the underlying modulated LIF model. The tensors W. T. u, R. Z of the RNN encapsulate parameters from the biophysical level (baseline weights, modulation sensitivities, release rates, interaction constants, etc.), often absorbing scaling factors and timescales during the reduction process. We frame this showing how even very low-dimensional modulatory system can model as a nigh-dimensional dynamics. This system likely works concert with other biological powerfully gate high mechanisms (like thalamocortical gating) to form a richer context space. Our goal is to show how volume transmission provides powerful, efficient, and interpretable means of broadcasting context.
2 Connections to Other Architectures
Interpretation as a Context-Factorized Hypernetwork
We define "context factorization" as the decomposition of a network's effective connectivity into a fast-varying neural activity and slow-varying modulatory state (the 'context'), Mathematically, the
26

---

effective synaptic weight from neuron j to is factorized as:
M
W (t) W
Tisk
"dynamics"
"context"
This allows the low-dimensional context vector rē()) to dynamically reconfigure the high-dimensional effective connectivity of the network. We use the word "factorization" to evoke the mathematical method of decomposing a function into a product of components here, the components are "context" (encoded in neuromodulator concentrations) and "dynamics" (encoded in the resulting synaptic connectivity patterns).
The derived mmRNN equations (Eq.
20 21 embody this concept of context factorization [4 18
The effective connectivity f (t) Waj + Σx (t) dynamically changes based the neuromodulator state rī(t). This state rīt)) evolving on potentially slower timescales (Eq. 21
acts as the "contett' that modulates the faster neuronal dynamics governed by F(()) (Eq. 20
This structure is equivalent 21 acts as the hypernetwork, dynamically generating (part of) the parameters (TGR) for the main RNN (Eqq. 20). Both the main network and the hypernetwork have recurrent dynamics [23). A defining feature of our e-nmRNN is its "endogenous" nature: the neuromodulatory state rā(() is generated by the network's 's own recurrent activity F(t) through the learned projection matrix R. This creates a closed-loop. self-modulating system. The advantage of this endogenous approach is that it allows the network learn to generate its own internal context states directly from its population dynamics, enabling it to autonomously infer the current context from an input stream and adapt accordingly. The term Ti represents tertiary interactions arising from a low-rank approximation to matrix Taylor expansion of the dynamics.
specific type of hypernetwork [8] The neuromodulator network (Eq.
This perspective connects nmRNNs other dynamic network architectures:
Switching Linear Dynamical Systems (SLDS): Neuromodulation rtt) can be seen as continuous version of the discrete latent state ((t) in switching Linear Dynamical Systems, smoothly interpolating between different dynamical regimes instead of abruptly switch-
ing(48) 48).
Fast Weights/ / Attention:
The dynamic modulation Ty (t) resembles fast weight pro- grammers or attention mechanisms, where temporal context dynamically gates modifies information flow from input to output 21 20).
Hypernetworks: The dynamic modulation (t) can be viewed linear readout hypernetwork which uses context module to generate the weights of the core network. Here, we note that the nmRNN adopts both recurrent dynamics and the view that the well- mixed neuromodulatory field is governed by volume transmission and therefore neuronal firing [18
High Dimensional Taylor Expansion: Neuromodulation via volume transmission can be viewed the next leading order contribution of Taylor expansion. Many-body physical systems with complex interactions rules can be expanded as:
4
f(z)
Σ AijTy
ΣΣ Tm OHigher order)
Binary Interactions
Three-bady interactions
The volume transmission equations can be viewed as a low-rank representation these three-body interactions proceeding the firing rate nonlinearity and thus represent the next leading order dynamics of the RNN.
Koopman operator theory (with polynomial basis): In the absence of the nonlinearity, this Taylor expansion can be viewed as Truncated Koopman linear operator acting over polynomial dictionary (49). A valuable way forward may leverage this powerful toolkit for fitting nonlinear dynamical systems.
Gating can viewed as the continuous "AND" gate where the corners of the space recreate the truth table for discrete input. Through combination of careful construction of the A (rank 2- tensor) and the (rank tensor), we represent "XOR" gates well forming the basis of discrete bit universal computation 50
Continuous logic gates:
27

---

We advocate for this model as tool not only to traverse levels of understanding in neurobiology, but also as scaffold to formally connect diverse approaches for learning in dynamical systems. As recurrent architecture, the e-nmR NN is inherently sequential during inference, which makes less parallelizable than models like Transformers. This represents a trade-off: we gain powerful inductive biases for online, continuous adaptation within single data stream at the cost of the massive parallelizability that makes Transformers so scalable for large-batch offline processing. We position the e-nmRNN learning, and brain-computer interfaces, where online processing is paramount.
architecture well-suited for applications like real-time control, agent-based
Connection to Costacurta et al., NeurIPS 2024
Despite differences in our respective derivations, we observe striking conceptual and experimental similarities to the valuable contribution by Costacurta al. [15]. which also explores the landscape of neuromodulation in recurrent neural networks. To clarify the precise connections and highlight key distinctions, we present a direct comparison of the final functional forms between the two model classes in Figure 7
Nole: using Einslein summation
Direct comparison in same language
AAgataee ΣΣ Appiat
Our work Both
Costacurta et al*
sigmaid (W2 + mpn rj+ + (pf a + ba) rj + uill
1; = o
Factorized context
Structured Variability
Ty Tn
ig =
RegTi Volume tranmission with firing
+
Zapry Coupling between nms
Direct aceess to inpot
(t(t) = (t) +
*After appropriate transformation into the dual representation
Figure 7: A direct comparison with Costacurta al. reveals both powerful similarities and key architectural differences. The most important distinctions between the two model classes include the low-rank nature of synaptic variability explored in their work, and the absence of direct access to context signal for our NM dynamics, which enforces an endogenous, closed-system interpretation of these dynamics.
We view our work as highly complementary, yet distinctly advancing the field through several orthogonal contributions:
Our model emerges from a detailed rate reduction of Lif- Lif-neuron dynamics incorporating neuromodulation via volume transmission, providing direct link to established biophysical principles.
Bio-inspired Derivation:
Novel Learning Rules:
We expand on the model class by deriving normative leading- order learning rules that explicitly link dynamics modification with context context-targetd credit assignment, crucial step for understanding how such systems might learn
Approximation of Gradient Descent: Crucially, we demonstrate that our e-nmRNNs can learn to approximate gradient descent and robustly encode vital learning signals such as Reward Prediction Error (RPF), providing a computational mechanism for online credit assignment biological and artificial systems.
Dynamic Generalization Tasks:
Our work explores performance on neunoscience-isspired compositional task sets and dynamic generalization tasks, which are distinct in both their formal structure and biological inspiration, showcasing the model's adaptive capabilities.
Strong Baseline Comparison: We conduct rigorous comparisons against strong base- lines, systematically assessing the e-nmRNN's capacity and compositional generalization performance.
28

---

Reverse Engineering Biological Specialization: The inherent interpretability of the nmRNN allows us to reverse engineer cell-type-specific neuromodulator dynamics and emergent cell specialization, offering a novel computational lens for probing neural compu- tation across complex task suites like Yang et al. (2019).
We hope the reader will agree that these contributions are deeply complementary and both serve to advance an important conversation in neuro-Al with their own strengths.
Detailed Learning Rule Derivations
we assume that it will look like a local version of gradient descent (in synapse and in time), we can find the gradient of the dynamics that will govern the normative leading order learning rules I1 37 12).
Let's define the task in terms of a simple MSE from target sequence. This target sequence can be represented multiple timescales signal via the Laplace transform:
(t) / dza(z)e
This makes our loss function equal to:
Σ (w (1) (0-)D((()))
where D the linear readout matrix that maps neural activity F(t) to the output y(t).
We can expand this to timesteps required to percolate information through the network by expanding:
rt)) r(t 1) + dt i(t 1)
and
(() = n(1 1) + dl n((
The first step is writing the error function as:
L Σ c(t)
where we assume that W 0.
(1) 1)
Th UT -7(())))
- - r(1 r((- 1) + dl
-r(1 1)
Here, e(t) is the instantancous error signal, and o is the nonlinear activation function for the rate dynamics (e.g., tanh ReLU-Tanh).
So then we need to plug in the n(t) as:
<(t) (t)= D ((r(e 1) + dt r((t 1) Σ Σ The (nx(t-)) 1)
+ dt( n((t - 1) + A(r((t - 1) + Zn(t 1111)
+ UI( 11)
where 9 is the nonlinear activation function for the neuromodulator dynamics.
29

---

We can expand this out to find:
<(t) y (t)
Dr(t- -
Σ Σ (1))(( --)n(-))):;(1111 1
+ dt
-r(t 1)+0 +
+ di²
-n(ll- ( ( 1) Zn(l - 1) 1(U - + m (2)((())))))))))))))>)))
By plugging these into our dynamics, we can find that our error can be written an expansion of dt. Local in time calculations and synapse follow the criteria of being to leading order in (ddt)). This then allows us to truncate this expansion leading order.
If we expand this to leading order in dt, we find:
(at))
T (t 1))rs(( (0)0(tt)))3((-11111 (1) =1 Σ &
<(t) (()) Dr(t - + dt (-((t11)+ 1)
This normative leading order learning rule will take the form of:
(dt) VTu & (A((t)"
where A(l) is the pre-activation input or "synaptic drive" to neuron's nonlinearity.
Interestingly, the presence of the kth neuromodulator should gate the update of the kth page of the three-body tensor. The gating via this multiplicative factor puts the neuromodulator on equal footing the error.
Next, the gradient of the input sensitivity takes the form of:
(ds) VaL D (A)))))(((
Again, the kth neuromodulator gates the input learning as well.
Next, we can study the gradient the neuromodulator dynamics with respect the error:
RRC (()(()
and the coupling matrix between neuromodulators leading order takes the form of:
(dt²)
VzL R D''(())((A()))) (A))) (B())n(()(()
These leading order learning rules could be sensible hypothesizes for how this joint system would evolve it was tuned to solve the problem of error minimization. (There is an analogous learning rule for minimizing the RL loss derived from Policy Gradients). This update rule is "context-localized" by (t) and "'emporally-localize" by using the instantaneous error signal e;()). While this truncates the full gradient through time, the recurrent state r(() and, critically, the slow neuromodulatory state ri(t) act as eligibility traces, carrying the necessary temporal context from the past. making the approximation potent enough solve complex tasks requiring credit assignment over intermediate timescales.
4 Statistical Significance of Gradient Alignment
In our analysis, we investigate the relationship between the dynamics carried out by the neuromodu- lators and the gradient of surrogate model. A surrogate model allows us calculate normative
30

---

learning signal, by way of ideal gradient to improve performance on the task. We accomplish this by locking the parameters of the e-nmRNN to coevolving RNN by setting: W Wij + Fi over every window of size = 100. We then use autodiff to calculate how W = - VW W C to best reduce the loss of the surrogate network. For the neuromodulator dynamics to "fall into alignment" with this surrogate model's gradient means that the neuromodulator states evolve in a direction that closely matches the optimal update direction prescribed by the surrogate model in math:
Tigkii (1)
-a W.C
learns approximate
Significance of Cosine Similarity in High Dimensions
To understand if the observed alignment significant, we will study observed cosine similarity of 0.15 between two vectors in nigh-dimensional space of D 128 16384 dimensions. In high- dimensional spaces, the distribution cosine similarities between two randomly and independently drawn vectors tends to concentrate sharply around zero. Specifically, for vectors whose components are drawn independently from a distribution with zero mean and finite variance, the expected cosine similarity is O. The variance of the cosine similarity between two random unit vectors in D dimensions is approximately I/D.
Our null hypothesis (Ho) is that the observed cosine similarity arises from two randomly and independently selected vectors, implying no true underlying alignment. Under this null hypothesis, the distribution of cosine similarities has mean of O and a standard deviation of approximately
= VV/D / & 0.0078. The observed cosine similarity of 0.13 is approximately ~ 16 standard deviations away from the expected mean of 0 under the null hypothesis. Such a deviation extremely improbable under this simple null model. Therefore, the observed and slowly accumulated cosine similarity of 0.15 constitutes a statistically significant alignment between the gradient of the surrogate model and the dynamics carried out by the neuromodulators, providing evidence against the null hypothesis that the vectors are unaligned and randomly oriented in this high-dimensional space. This combined with the learned nature of this alignment suggests numerically, that e-nmRNNs not only can, but do learn to approximate gradient descent through gradient descent similar to that observed in meta-optimizers [2] and transformers 20)
Dynamical Regimes and Bifurcations
Traversing Bifurcations: Multiplicative vs. Additive Modulation
Dynamical systems often exhibit bifurcations, where small parameter change leads to qualitative shift in behavior (e.g.. from a stable fixed point oscillations). Context factorization provides
distinct mechanism for navigating these critical transitions compared to traditional input modulation,
and these mechanisms leave different measurable signatures.
We can illustrate this difference using simple 2-neuron RNN model:
dri
dt -r tanh(n + Wi2r2) r3)
(22)
dr
di -2 tanhi(((Wairr W22r2) ra)
(23)
Here, n represents multiplicative context factorization (gain modulation), while r3 represents additive context input (e.g., from dedicated context neuron). Both can drive bifurcations, but how they do so differs fundamentally.
Multiplicative Modulation (n): When = 0, the origin (0,0) is always a fixed point due to the odd symmetry of tanh. Linearizing around the origin reveals eigenvalues À (2n 1) + in for the specific W matrix used in simulations (W11 W22 = 2, W = -1, W = 1). A Hopf bifurcation occurs when Re(A) = which happens precisely at n = 1/2. As n increases through 1/2, the origin transitions from stable spiral to an unstable spiral surrounded by stable limit cycle (oscillation)(24) Critically, this bifurcation happens without shifting the fixed point away from the origin. The bifurcation preserves the system's 's symmetry around (0, 0).
31

---

P === -r + (n(() [Wih + Wists + Uir)((()
À = = [Warr + Wayrsl + Un))(())
NM-lkke
direction
Imput
Val. Loss
Training across bifurcation
P
Output @ cheokpoint
111111 17
logT0/cls)
Binned
Training steps
Figure 8: Studying the role of gating in the Hopf bifurcation in tnnigg and in training. A) We study these two 3 dimensional dynamical systems. Black text is present in both. Blue represents context factorization and Red represents the presence context storing neurons. B) We show the difference of how nm-like and input tuning across the two Bifurcations looks for the tanh() nonlinearity. Flow directions are signaled by the color of the streamline denoted in the circular key to the right. C) We show the evolution of the signal output across training and show that the "epiphany" moment in the validation loss corresponds to the emergence of the oscillations suggesting the dominant challenge is the formation of the limit eycle.
Additive Modulation (r3): When r3 # O, the origin is no longer a fixed point (tanh(r3) * 0). The fixed points (F1,7z F2) satisfying n; Lanh(n(WF); + rs) shift Y3 changes. Stability is determined by the Jacobian evaluated at these non-zero fixed points:
(2) = (1
(1-
(1 (1- (1
(2-
Changes in r3 alter (F1, f2) and thus the Jacobian elements, potentially driving eigenvalues across the imaginary axis. However, any bifurcation induced by necessarily occurs at non-zero operating point (F1, F2) and breaks the symmetry seen when T3 = O. Depending on the specific path traced by (F1,52) r3 might induce Hopf saddle-node bifurcations.
Distinguishing Mechanisms (Measurable Differences): These differences provide experimentally testable predictions (visualized in Figure IB):
key distinction whether the mean activity (average F1, f1, F2) shifts significantly as the system crosses a bifurcation. Additive modulation (ra) inherently involves an operating point shift, while multiplicative modulation (n with T3 0) can induce bifurcations (like the Hopf bifurcation at the origin) without necessarily changing the mean activity level of that specific fixed point.
Operating Point Shift:
2. Symmetry Treservation/Breakine Bifurcations driven by multiplicative gain modulation in a system with appropriate symmetries (like r3 0 here) tend to preserve those symmetries (e.g., limit cycle emergence centered around the origin). Additive inputs break this symmetry, leading to asymmetric dynamics relative to the origin.
Observing whether a qualitative change in network dynamics (e.g., onset oscillations) accom- panied by a shift in the mean firing rates provides potential signature to distinguish between gain modulation (context factorization) and direct input modulation as mechanisms for context- dependet dynamic switching in biological or artificial networks.
32

---

Bifurcation Analysis of Rank-2 RNN with Multiplicative and Additive Context Modulation
We analyze 2-neuron Recurrent Neural Network (RNN) whose dynamics are given by the following system of coupled ordinary differential equations:
dr
= 11 + @(n(Wi1r1 + W12r2) + r3)
(24)
dt
dr2 r2 + @(n(W21r + Wx2r2) + r3)
(25)
dt
where:
ra (t) and r2()) are the firing rates of the two neurons.
((x)=
tanh(r) is the non-linear activation function.
Wn 221
W12 W22
-1
W
is the synaptic weight matrix.
2
is multiplicative factor representing gain modulation context factorization. It scales the synaptic inputs before the nonlinearity.
P3 is additive factor representing external input the influence of 'context neuron', It shifts the input to the nonlinearity.
Both n and r3 act as control parameters that can drive bifurcations in the system's dynamics. We aim understand the distinct effects of varying n versus varying r3.
Fixed Point Analysis
Fixed points F2) of the system solutions to dri 0 and dea 0. This yields the equations:
F <(n(W11F1 W12F2) + r3)
(26)
F2 c(n(W21F1 W22 + r3)
(27)
Let and define the input vector nWr
ri
T3
Then the fixed point equation is
12
(T))
Symmetry and the Origin
Consider the case when the additive input r3 = 0. The fixed point equations become:
tanh(n(W + Wi2F2)) F2 Lanh(n(W2217 + W22r2))
Since (O) 0, the origin (21, (0,0) is always fixed point when ra regardless of the value of the multiplicative factor n.
Now consider the case when r3 # 0. The origin (0, 0) is fixed point only if:
= tanh (0) + r3) = tanh(r3)
0 tan + r3) tanh(rs)
This requires F3 = 0. Therefore, the additive factor 73 breaks the symmetry that keeps the origin fixed. An additive input rs # 0 shifts the location of the fixed point(s) away from the origin.
Measurable Difference 1: The mean activity (operating point), corresponding to the fixed point location (F1, will generally shift when bifurcation is induced by changing 13. In contrast, if bifurcation induced by changing n while r3 = 0, the origin (0, 0) remains a fixed point, although its stability may change and other fixed points may appear/disappear symmetrically around the origin.
33

---

Stability Analysis and Bifurcations
To analyze the stability of fixed point (f1, F2), we linearize the system around it by computing the Jacobian matrix J:
0t.
dru dr
=
(P1.P2)
Let ø1 = n1 + W1272) + ry and D2 = (2111 + W2272) + Recall d'(x) = sech?(x) =
1 - taahhnnna (x)) The partial derivatives are:
Ør1
= 1 + (x11) -n
dr
0r d'(1)) .112
0r2
0rg o' (222 m121
0r1
0r 1 + 0' nW
Ör2
At fixed point (F1, F2), we have f1 ((²1) and ((B2). -1 The Jacobian at the fixed point (F1, F2) is:
(-
(-1 1
(1-
((
(1 -1-15 2タ n221
-1 (11-55 W2)
(1 p W1
(1 (1- 1-2
((1 =
W2
(1 (1-r5 (-rr))22
where I is the identity matrix.
Stability of the Origin (when r3 = 0)
When ra = O, the origin (21, = (0, 0) is always fixed point. Its stability is determined by the Jacobian evaluated at (0, 0):
(1 (1-0")W12) Wn
J( 0)
- nW -
W2
(1-02)Wzz
Substituting the values for W:
2 -1
0 -
2n - 2n - 1
J(0,0) 0) =
2
11
The eigenvalues À are found from the characteristic equation det( 0) A7)
(2n - 1 A)2 - (n)
(2n 1 - 1) -"=
x - (2n - 1) = =v Łin
X = (2n 1) + in
bifurcation occurs when the real part of the eigenvalues crosses zero. Here, Re(A) 2n
If 2n - < 0 (i.e., n < 1/2), Re(A) < 0. The origin is a stable fixed point (specifically, a stable spiral since Im(A) in + for # 0).
If 2n 1 0 (i.e., n > 1/2), (e())= > 0. The origin is an unstable fixed point (an unstable spiral).
If 122n11 11-1= = 0 (i.c., n 1/2), Re(A) (A) 0. The eigenvalues are purely imaginary (A +i/2). This indicates a Hopf bifurcation at n = 1/2.
As increases through 1/2 (with giving rise to stable limit cycle surrounding the now unstable origin. This bifurcation occurs without shifting the location of the fixed point itself (it remains at (0, 0)).
0), the stable spiral fixed point at the origin loses stability,
34

---

Effect of Additive Input (r's + 0)
When 13 # O, the origin is no longer fixed point. The fixed points (f1, F2) must be found numerically
or graphically by solving Eqs. 26 27 The Jacobian must be evaluated at these non-zero fixed points:
(1-19) 1 (
(1
J(F F2) = (1 (1- 22) (1-r2)
As ra is varied (keeping n fixed, e.g.. n = 1), the location of the fixed points (71, F2) will change. The terms (1 Ff) in the Jacobian will also change. Bifurcations can occur when the eigenvalues of (F1,, F2) cross the imaginary axis.
Consider the trace and determinant of the Jacobian:
Tr(J) (-1 + (-
=n((11 (1((1- (1 (1- 22)) 2
det(J) (-1+ (-
- 222) ((1
A Hopf bifurcation occurs when Tr(J)= and det > 0. A saddle-node bifurcation (where fixed points appear or disappear) occurs when det(J)= = 0.
Varying changes (71, F2). which in turn affects both the trace and determinant. This means can potentially drive both Hopf and soddle-node bifurcations, depending on the value of n and the specific path taken by (F1, F2) as changes. Crucially, any bifurcation induced by r3 will be associated with a fixed point (F1, F2) * (0.
Measurable Difference 2: Bifurcations driven by the multiplicative factor (with r'3 = 0) preserve the symmetry around the origin. For instance, the Hopf bifurcation
1/2 happens precisely at the origin. Bifurcations driven by the additive factor 73 occur at non-zero operating points (Fi, F2) and generally break the symmetry observed when r3 O. The type of bifurcation (Hopf vs. saddle-node) can also differ depending on whether n or is the control parameter.
Summary of Measurable Differences
To distinguish between bifurcations driven by multiplicative gain modulation (n) versus additive input (ra):
1. Operating Point / Mean Activity: Monitor the average values of ri and 72-
Varying (with 0): The origin (0. 0) remains fixed point. Bifurcations (like Hopf at = 1/2) occur centered at the origin. Other fixed points may appear/disappear symmetrically.
Varying r3: The fixed point(s) move as r3 changes. Bifurcations oceur non-zero operating points (F1, F2). The mean activity changes significantly across the bifurcation.
2. Symmetry: Observe the phase portrait.
Varying n (with F3- = 0): The dynamics (fixed points, limit cycles) should exhibit symmetry, c.g., if (F1, is fixed point, (-F1, might also be (depending W. though not guaranteed for non-zero fixed points with this specific W). The Hopf bifurcation occurs symmetrically around the origin.
Varying r3: The symmetry around the origin is broken. Fixed points and limit cycles will generally not be symmetric.
3. Bifurcation Type at Origin:
Varying (with r3 = 0): Specifically leads to a Hopf bifurcation at the origin when 1
Varying r3: Cannot induce a bifurcation exactly *at* the origin (as it's not fixed point for r3 # 0). Bifurcations occur elsewhere in the phase space.
By measuring the mean activity levels (F1, and analyzing the symmetry of the dynamics as the control parameter varied, one can distinguish whether a bifurcation primarily driven by multiplicative (gain-like) or additive (input-like) contextual factor.
35

---

Deep Reinforcement Learning Experiments
For the reinforcement learning tasks, utilized a block-switched two-armed bandit task where reward probabilities invert stochastically (Figure 4A). To address the potential confound of periodic switching, we ran new experiment where the high-reward arm switched at stochastic, unpredictable intervals. The agent adapted robustly, confirming it learns genuine model of the task state and is not merely entraining to a fixed frequency. The agent must learn to track the hidden block state (i.e., which arm has high probability) and adapt its actions to maximize reward. Training e-nmRNN agents was performed using Proximal Policy Optimization (PPO), standard deep reinforcement learning algorithm.
To analyze the RPE encoding, Maximum Likelihood Estimation (MLE) was used to fit a Q-learning model to the generated action and reward sequences from the trained e-nmRNN agent. The RPE signal was then extracted directly from this fitted Q-table. Finally, single degree-of- decoder was trained to map neuromodulator states to various behavioral features, including RPE, choice, reward, and entropy.
linear
Table 2: Key Hyperparameters for Reinforcement Learning Task
Parameter
Value/Range (PPO/TPE)
Notes
Batched Environments
16
Fixed
PPO Learning Rate
7x10-5
Fixed
PPO Clip Epsilon
Fixed
Discount Factor (7)
0.95
Fixed
Generalized Advantage Estimation (A)
0.95
Fixed
Sequence Length
200
Fixed
Hidden Units (RNN/e-nmil (NNN/c-nmRNN)
64
Parameter-matee
Number of NMs
4
Fixed
NM time constant (T,) Activation Function
0.1
Optimized by Hand
ReI.UTaahh
Fixed
The model fitting uses the public github methods from the Allen Institute for Neural Dynamics to fit an MLE to the action sequences of the nmRNN agent.
[language=Python, basicstyle=, breaklines-aee tnes= forager ForagerCollectionn) retat Bari2019", seed
rer.chieeeiissooorrrrrrrssssoo fut.aaaaooooo...oooaaaaaraaaa verridd
"so I [0, 100), clamp,arams
=
"biasL" 0, wargs
dirlwworkers 4, disi Truc, seed
None,)
Multitasking Benchmark Details
Table 4: Summary of Neuroscience-IIspiree Cognitive Tasks from Yang
et al. (2019).
Task Name (Abbreviation)
One-Sentence Description (Inspira-
Task Dynamics (Timing)
tion)
Go Task Family Go (Go)
A pro-response task requiring a re- sponse in the direction of single stim- ulus (modality or 2) after a go cue. re-
Stimulus is shown (Tstim U5500, 1500) ms) before the fixation cue goes off.
Reaction-time Go (RT Go)
A pro-response task requiring sponse in the direction of a single stim- ulus (modality or 2) as soon it ap- pears.
Response
cued
by
onset (Tatim U500, 2500)
stimulus
ms); fixation cue never goes off.
Continued on next page
36

---

A memory-guided saccade task to re- member a brief stimulus and respond in its direction after delay and go cue. Table 4: Summary of Neuroscience-IIsppree Cognitive Tasks from Yang et al. (2019).
Task Name (Abbreviation)
One-Sentence Description (Inspira- tion)
Task Dynamics (Timing)
Delayed Go (Dly Go)
A brief stimulus is followed by a vari- able delay (Tdetay! ((200, 400, 800, 1600)) ms).
Anti Task Family Anti-response (Anti)
An anti-saccade task requiring re- sponse the direction *opposite* to
Same as Go task (T stmmm U(500, 1500) ms).
single stimulus (modality 1 or 2) after go cue.
Reaction-time
Anti (RT
An anti-saccade task requiring re- sponse in the direction *opposite* to single stimulus (modality 1 or 2) as soon as it appears.
Same as RT Go task (Tstim ms).
Anti)
~
U(500, 2500)
Delayed Anti (Dly Anti)
An anti-saccade task remember brief stimulus and respond in the *oppo- site direction after delay.
Same task U[2000, 400, 800, 1600)) ms).
Diy
Go
(Tactay)
Decision Making (DM) Family Decision Making (DM I)
A perceptual decision-making task to choose the stronger of two simultaneous stimuli, both in modality 1.
Two stimuli shown simul- tancously for Tatim U{4400, 800, 1600}) ms. Two stimuli shown simul- taneously for Tatsmi U (400, 800, 1600)) ms. Two bi-modal stimuli shown simultaneously for Tstim U ((400, 800, 1600)) ms.
Decision Making 2 (DM 2)
A perceptual decision-makin choose the stronger of two simultaneous stimuli, both in modality 2.
task to
Context-dependent DM (Ctx DM 1)
A context- context-dependent computation task choose the stronger stimulus in modality while *ignoring* modality 2.
Context-dependent DM 2 (Ctx DM 2)
A context-dependent computation task choose the stronger stimulus in modality 2 while *ignoring* modality
Two bi-modal stimuli shown simultaneously for T Tstim U ((400, 800, 1600)) ms.
Multi-sensory DM (Mult- Sen DM)
A multi-sensory integration task to choose the stimulus with the stronger *combined* strength from both modali- ties.
Two bi-modal stimuli shown simultaneously for T Tetmm U ((4000, 800, 1600}) ms.
Delayed Decision Making (Dly DM) Family Delayed DM 1 (Dly DM 1) A parametric working memory task to compare the strength of two sequential, brief stimuli in modality 1, separated by a delay.
S1 (300 ms). delay (Tdetay1 ~ Uf 1600)) ms), then S2 (300 ms) in mod 1.
Delayed DM 2 (Dly DM 2)
A parametric working memory task to compare the strength of two sequential, brief stimuli in modality separated by delay.
S1 (300 ms), delay (Tdelay) ~ 1600))
ms), then S2 (300 ms) mod 2.
Ctx Dly DM (Ctx Dly DM 1)
A novel task (working memory + gat- ing) to compare sequential bi-modal stimuli based *only* on modality 1.
SI (300 ms), delay (T (edetag1 ~ U600).6000)
ms), then S2 (300 ms); both bi-modal.
Continued on next page
37

---

Table 4: Summary of Neurosciencee-Ispprre Cognitive Tasks from Yang
et al. (2019).
Task Name (Abbreviation)
One-Sentence Description (Inspira- tion)
Task Dynamics (Timing)
Ctx Dly DM (Ctx Dly DM 2)
A novel task (working memory gat- ing) to compare sequential bi-modal stimuli based *only* on modality 2.
S1 (300 ms), delay
(Taetay1 ~ U (7.1..16000 600))
ms), then S2 (300 ms); both bi-modal.
Multi-sensory Diy
DM
A novel task (working memory + in- tegration) to compare the *integrated* strength of two sequential bi-modal stimuli.
S1 (300 ms), delay (deelgg1 ~ Ut 1600))
(MultSen Dly DM)
ms), then S2 (300 ms); both bi-modal.
Matching Family
Delayed Match-to-Sample
A visual working memory task to re- spond to S2 *only if its direction is the *same* as S1, regardless of modality. A visual working memory task to spond to S2 *only if* its direction is *different* from S1, regardless of modality.
Brief SI, delay (Tastoyl U((...1000)) ms), then brief S2.
(DMS)
Delayed Non-Match-to-
Brief S1, delay (Tdelay1 U(f...1600) ms), then brief S2.
Sample (DNMS)
Delayed
Match-to-
A visual categorization task to respond to $2 *only if* its direction belongs to the *same category* S1.
Brief S1, delay (Tactay) U((_..1600)) ms), then brief S2.
Category (DMC)
Delayed Non-Match-to-
A visual categorization task to respond S2 *only if its direction belongs to *different category from S1.
Brief SI, delay (Tdetayl U U((...6000)) ms), then brief S2.
Category (DNMC)
We evaluated the e-nmRNN on a suite of 20 neuroscience-inspired multitasking benchmarks from the established suite by Yang et al. (2019), which covers wide array of decision-making and cognitive tasks. This benchmark suite is designed to probe flexible computation and generalization across diverse cognitive demands. The network is provided one-hot task ID at the beginning of each trial. The crucial finding is how the network learns to use and maintain this information. The architecture's inductive bias encourages this context to be encoded and dynamically maintained within the neuromodulatory state rī(t). As shown in Figure 5C, a linear decoder trained on these neuromodulator concentrations can reliably identify the current task long after the initial cue has passed, whereas decoder trained on the neural activity F(t) cannot. This demonstrates that the e-nmRNN learns represent the task context in its endogenous modulatory state, using it to shape the circuit dynamics appropriately for the task at hand. A parameter-matcher standard RNN was also trained on the multitasking benchmark. The results show that our spatially-embeddees e-nmRNN achieves comparable or superior performance across the 20 tasks, demonstrating that the inductive biases of our architecture provide tangible value on this complex benchmark. The specific task parameters, input/output structures, and success criteria for each individual task in the suite detailed the publicly available code repository (details Section L0P.
8 Detailed Analysis of Emergent Biological Structures
Our analysis of trained e-NmRNNs reveals several emergent properties that mirror observations in neurobiology, providing insights into the model's 's interpretability and biological plausibility.
8.1 Modularity and Cell Clustering
Training on multitasking benchmarks promotes the emergence of functionally specialized modules within the e-mmRNN, as well as hierarchical cell clustering (Figure 5D). This modularity is supported by two key observations: 1) the formation of "Sherringtonian eircuits" among presynaptic partners 31. and 2) the emergence of distinct, modular brain regions in biological systems. Our clustering
38

---

Table 3: Experiment Parameters for the agents shown in the main text.
Parameter
Value
Parameter
Value
env_name
policy_use__ saaly__se__ace
true
exp_label
rl2
policy_lambda
0.95
max_episode_stepp
200
policy_algorithm
a2c
max Lixx_rollouts_pe Becbouussperrtas__ seed time as _tatt
policy_critic crt__rritic__sss_____ llss_lossseel__
0.01
73
policy_entropy
0.3
false
policy_optimizz
adam
deterministic. _eeecttioneeeni eesults_log_dir log_ intervll save_inte Lnveiintrvall
false
policy_eps
le-08
/scratch/
policy_Ir
0.0007
500
policy_anneal_i Ir
false
1000
Seicc__mx_ggrad___ norm
0.5
awve_intrmmdiite..c models
false
salicy_us__activig reeguularraatiiiiinnoon
true
eval_interval
500
bolicy_activit____lossccco NMd
100
eval_ids
[
true
num _vve envs envs vis interval num oum_updates Lrcc___um_stesspee____
300
nNM state dim
4
500
1
50000.0
input state dim for _or_yolcy
0
200
action iitoo_sppaee_ypp
Discrete
num sum_processes deterministic_poliyy
16
action_dinn
2
false
reward dim
1
shared_mm mn
true
device
cuda
layers_best werrs_bffoe__mm_
[
mn inn_hidden_dim
64
ayees_after__m _cce_ttpeee_ee_eeeeee_______e_ee
0
vanilla
cetioon_embed action embed dim state_ siiteeeemeedd_eeee dim
0
embed reward_embed _im
policy_net, Beryy_ee__ctee mnt_activvttion function
relu-tanh
policy_net, _e____ee_____ wil_neeiiiialzzattinii. metieiiiiatiommmeheddd method
norme
action_pred type
bernoulli
Sodden_nnise__t
0.002
policy_gamma
0.95
methods, detailed in Section 10 group neurons based on their connectivity patterns and activity profiles, revealing clear modular organization that aligns with functional specialization.
8.2 Cell-Type Specialization in Neuromodulator Release
We observe striking specialization where individual units learn to predominantly promote the release of a single type of neuromodulator (Figure 5E). This differentiation is functional and learned, not hardcoded. A unit's activity promotes the release of specific neuromodulator the corresponding column in the learned release matrix R becomes selective for that modulator. For instance, if the j-th column of R.y, becomes approximately one-hot vector [1, 0, 0, ...]". then neuron j's activity rj primarily drives the dynamics of the first neuromodulator, neuromodulator are then determined by how its corresponding entries the learned tensor T gate synaptic transmission throughout the network to solve the given tasks. This emergent property provides a concrete computational model for how diverse neuronal components contribute overall system function. We also found that units specializing in releasing specific neuromodulator form no notable spatial clusters, providing new evidence strengthening our claims of emergent, biologically- reminiscent organization.
The functional effects of this
39

---

8.3 Spatial Autocorrelation Analysis of Neuromodulator Loadings
To quantify the spatial "mixedness" of the four neuromodulator (NM) loadings across the cell grid, we computed the spatial autocorrelation using Moran's 's I. This statistic measures the correlation of variable with itself in spatial terms. The global Moran's 's is defined as:
N N N N 2i=1 j=1 Wij (2 5)(r 5)
I =
So
E 8)²
where N is the number of cells (256), is the NM loading for cell i, E is the mean loading for that NM, and wiy is the spatial weight between cells and j. The spatial weights matrix was constructed from the Delaunay triangulation of the cell centers (r), such that Wij if cells i and are neighbors (i.e., connected by an edge in the triangulation) and Wij = otherwise. So the sum of all weights, Ee E, Waj. A value of I O indicates positive spatial autocorrelation (clustering or low mixedness), 7 R 0 indicates no spatial autocorrelation (a random, well-mixed pattern), and 0 indicates negative spatial autocorrelation (a checkerboard-like dispersed pattern).
U.A Ouaden
Figure Localizied Spatial Autocorrelation cluster maps for the four neuromodulator loadings (NM 0-3). Global Moran's 's analysis did not reveal significant spatial autocorrelation for any neuromodulator (NM0: p = 0.437, NMI: p = 0.216, NM2: 0.251, NM3: p = 0.465).
To visualize the local spatial structure, we computed the Local Moran's (LISA) statistic for each cell d:
I
F4 S
Σ
Wis (I, T)
where S? is the variance of x (excluding cell i). This analysis identifies spatial clusters and outliers, as shown in Figure The map categorizes cells based on their 4 value and its statistical significance (p-value):
High-High (Hotspot): cell with high NM loading surrounded by other cells with high loadings.
Low-Low (Coldspot): A cell with a low NM loading surrounded by other cells with low loadings.
High-Low Low-High (Outliers): A cell that is significantly different from its neighbors.
40

---

Not Significant: A cell whose local pattern is not statistically different from random distribution.
8.4 Timescale Separation in Neuromodulator Dynamics
The neuromodulator dynamics within the e-nmRNN spontaneously learn to evolve on distinct, separated timescales (Figure 5F). Our autocorrelation analysis, performed on the time series of individual neuromodulator concentrations, reveals a factor of approximately 8x difference between the fastest and slowest evolving neuromodulators. This heterogeneity timescales reflects the value of functional specialization, allowing different neuromodulators to encode information at different temporal resolutions, which consistent with biological observations regarding brain state and behavior. Details of the autocorrelation analysis, including calculation methods and statistical evaluation, are provided Section 10
8.5 Context Encoding within Neuromodulator Dynamics
Beyond compositional generalization, the neuromodulator state ri(() robustly learns to represent task context (Figure 5C). In experiments involving dynamically changing environments, such as the bloek-switched two-armed bandit task (Figure 4A), the NM concentrations dynamically track the hidden block state (Figure 4C, D). This Iearned context encoding represents compelling computational hypothesis for animals trained across multiple tasks or contexts, especially with the advent of emerging, temporally-resol concrete, testable prediction: during multitask learning, ambient NM concentrations in relevant cortical area should become decodable for task context, hypothesis that becoming directly testable with new imaging techniques (e.g., GRAB sensors).
neuromodulator imaging technologies. This makes
8.6 Reward Prediction Error (RPE) Encoding in Reinforcement Learning
In reinforcement learning tasks, specifically the dynamic foraging task, the neuromodulator state only encodes task context but also learns to represent critical learning signals such as the Reward Prediction Error (RPE) (Figure 4E, G). This a key finding: demonstrates how the e-nmRNN can leverage its neuromodulatory system to represent a critical learning signal, consistent with biological observations of dopamine activity correlating with RPE. This RPE encoding, combined with the (x 4-factor learning rule (Section 3.1), provides a concrete mechanism for RPE-gate online credit assignment, facilitating rapid adaptation to changes in the environment and contributing to the network's 's ability to "learn how to learn" in reinforcement learning context. This also leads to a testable prediction: in RL settings, some component of the neuromodulatory state space should explicitly encode RPEs, consistent with known roles for dopamine and norepinephrine.
Experimental Details and Hyperparameters
This section details the experimental setups and hyperparameters used for all reported results, includ- ing compositional generalization, multitasking, and reinforcement learning tasks. All experiments were conducted in a cloud environment utilizing AWS, requiring less than 64 GB of RAM and CUDA-enabled GPU for execution.
9.1 Sine Curve Meta-Learning
To assess the model's ability to learn and generalize across family of structured relationships. we employed simple meta-learning sequence generation task. Each input-target sequence pair is sampled from a family of sine curves defined by three parameters: amplitude (A,), frequency (w;), and phase (o:). The target sequence is given by:
y (1) = A, sin (wit + dx)
The input to the network at each time step is three-dimensional vector containing these parameters:
A,
I(t)
Wi
41

---

Importantly, due to the infinite size of this dataset, we opt to operate in continual learning regime for this optimization where each sample is both the first and last time the network sees that exact sample. For this reason, we report results not in terms of number of times through the dataset, but instead, number of batched gradient update steps.
The challenge of learning to pair the input parameters with the output sequence should not be underestimated. Indeed, if sample from the full, uniform distribution parameters. the networks (all model classes) routinely fail to learn to oscillate. Thus, to focus on operating on both sides of the Hopf Bifurcation, we focused on fixed distribution on and Φ to sample from A, max(4[-11,1..0
We began training with TPEsampler hyperparameter tuning. Our objective function is the minimization of the held-out loss after 1000 batched gradient update steps.
Next, we applied these locally optimal learning rates to each model class and trained the networks across samples. Batch sizes are held fixed (B = 16) across models to ensure that number of batched gradient updates are compared equal footing. We find that under these conditions, the e-umRNN takes many fewer update steps to achieve high performance across this domain and find that the exponentially weighted spatially embedded RNN improve batch update efficiently when paired with the e-nmRNN dynamics. This phenomena, combined with the interpretability advantages of ppatially-embeddee network, motivate continued use throughout this work.
9.2 Compositional generalization
This task extends the concept of matrix multiplication a setting where sequences are mapped the amplitudes of other sequences. Instead of treating inputs and outputs static vectors matrices, we view them as temporal structures, allowing for:
Representation Learning:
The input sequences encode structured representation, and the network must infer a decomposable basis from these representations.
Function Approximation: The network learns to reconstruct an output sequence from set of basis functions.
Compositional Generalization: If trained correctly, the model should generalize to new sequence compositions beyond its training set.
Mathematical Formulation of the Task
We define the input-output mapping as:
T(1) = Σ 0RP4 (1)
y² (1) Σ (m)(l))
where:
Mu (1) are the input basis functions.
m/ (t) are the output basis functions.
ag are the unknown coefficients that want the model to infer from the input sequence and use that inference to generate the associated patterns in the corresponding weights.
The model must learn a function fo that maps input sequences to output sequences by implicitly recovering ag:
((I(t))=v')
x
Generalization & Zero-Shot Performance
To test generalization, we evaluate the model on unseen compositions of input sequences:
1. Train the model on a fixed set of basis functions.
42

---

2. Introduce new linear combinations of basis functions during testing by holding out com- binations of samples from training in the coefficient hypereube. We do this using filtered uniform sampling to split "quadrants" of the hypereube into a validation set.
3. Measure validation performance on reconstructing previously unseen compositions.
the model generalizes well, this suggests it has learned an effective representation space that allows for compositional inference.
One signature learning this representation well the ability to decode (using weakly expressive decoder) the actual coefficients of each sequence from the network's s hidden units (or nm fields in the case of the e-nmRNN). We add this analysis to our study of the capacity better understand the failure to generalize with growing dataset complexity.
Two types of scaling
One of the great strengths of this simple task set is that scale both the complexity of the data set and the amount of data with the only cost being compute time. This allows us to ask how do each of the models selected for baseline characterizations scale with:
1. Scaling with the amount of data available train the model allows us to test for the "efficiency" of the representation learning
2. Scaling the complexity (by way the number of basis patterns used in the composition, K) allows to measure the practical capacity of the model.
In the work, explore both of these scaling relationships but only report the results for the capacity due to lack of space. The e-nmRNN's higher capacity makes more prone to overfitting on this smaller dataset compared validation loss at high task complexity (K > 15). A future research topic will investigate effective forms of regularization avoid this. Future work will explore an analytic explanation of these numerical experiments.
GRU, which can be seen in the generalization gap between training and
Reproducibility parameters for compositional generalization
For the multitasking benchmarks, all models were parameter count matched at approximately 50k trainable parameters and tuned on compositional generalization (K=8). Hyperparameter optimization was performed using -te-Sttue Parzen Sampling with 100 samples for all models. Models were trained for 1000 epochs, and the best variance explained on the validation dataset was extracted for performance evaluation. Performance scaling with "Task complexity" (K ranging from I to 20) was explored.
Table 5: Key Hyperparameters for Compositional Generalization Task
Parameter
Value/Range (TPE)
Notes
Learning Rate
[10-4 10 10-2
Optimized via TPE sampler Fixed
Optimizer
Adam
Batch Size
16
Fixed across all models
Hidden Units (RNN/GRU)
256/128
Parameter-matched
Neurons (e-nmRNN)
128
Parameter-matched Fixed
Number of NMs
NM time constant (Tx)
0.11,
Optimized
Weight Regularization (L2)
10-0,10041 10-4)
Optimized
Activity Regularization (L2)
Fixed
Activation Function
ReLU-Tanh
Fixed
9.3 Multitasking Benchmarks
On the 20 Yang19 tasks, we conducted a single study in the manuscript: what does the solution space high performing e-nmRNN look like? Do we observe biologically aligned phenomena simply by training the umRNN sufficiently rich set of tasks was seen in 27 32 28
43

---

Table 6: Key Hyperparameters for Multitasking Benchmarks
Parameter
Value/Range (TPE)
Notes
Learning Rate
[5 10-6, 5 10 10-3)
Optimized via TPE sampler
Optimizer
Adam
Fixed
Batch Size
32
Fixed
Hidden Units
224
Parameter-matched ( (0kk parameters total)
Number of Neurons
128
Fixed
Number of NMs
4
Fixed
NM time constant (7n)
[0.01, 10]
Optimized
Regularization (L.2)
[10- 10
Optimized
Activation Function
ReLU-Tanh
Fixed
Dale's s Law
Enabled
For spatially embedded networks
Spatial Embedding Lengthscale
10% of domain
For spatially embedded networks
10
Reproducibility Details
To ensure full reproducibility of all experimental results presented in this paper, will provide access to the codebase, including training scripts, evaluation procedures, and data generation methods. The code will be made available GitHub repository post acceptance, and a link for is provided for submission review. Additionally, we plan to provide associated Docker instances hosted on CodeOcean to facilitate ready and consistent reproducibility across different computing environments.
The code repository will include:
Source Code: All Python scripts for defining the e-nmRNN architecture, baseline models (Vanilla RNN, GRU, Transformers, HiPPO-LegT, Oscillatory RNNs), training loops, and evaluation metrics.
We provide a Dockerfile ile environment specification detail- ing all required software libraries and their exact versions (e.g., PyTorch, NumPy, Matplotlib, Optuna, gymnasium).
Environment Configuration:
Data Access and Generation: Scripts for generating the synthetic datasets used in the com- positional generalization tasks (sine curves and dictionary mapping). For the multitasking benchmarks and RL tasks, instructions or links to access the generated datasets are provided, along with preprocessing scripts.
Hyperparameter Configurations: Detailed configuration files for the optimal hyperparam- eters found during the TPE sampler optimization for each experiment, complementing the tables in Section 9
Clear instructions, including exact command-line arguments, to execute the training and evaluation for all results presented in the main text. We present these with saved output as jupyter notebooks for ready inspection.
Reproducible Commands:
Summary of code structure: please see below for detailed summary of code structure focusing the minimal path toward reproducing notebooks and environment initialization (denoted by a *)
We commit to releasing the assets under CC-BY 4.0 license. All aspects related to anonymity, as per NeurIPS guidelines, will strictly adhered to in the publicly released version of the repository.
We make our code available at https: / thub. com conm/stoormbll// Batt-mmRN___aMM nnw.//N2eeerrpp22555551 Please note that we highlight only the notebooks to run access the demos but have attempted to provide a complete and blinded version of the code needed to run it.
Final_ PiiallLMuurr1P_euubiissssooo
* REPRODDCINGG.m
Fig2 Hopf MetaLearning
Dockerfile
re5reffctor:
The full code needed to reproduce gradient alignment
44

---

Run the self-contained notebook: Refactor_ www.:roooos.oeiiaaaaaarrrrraaaaa 1pynb
Folder also includes analysis files of pre-trained logs
Fig3
Compositional Generalization
Environment/Dockerfite
Hyperparameter tuning can be hyperparameter cuning-Demo.. ipynb To train all models in model.py across the task cending complexity run [caution:
recreated by
running
with
24 hours
execution
time]:
Fig4
RL Bandit
PPO train, ttrii_nmRMNi mmRNN
Givironments/Dcckeriil
Folder provides access to full repo and some trained example networks.
To train
e-nmRNN agent please specify local paths and run [warning execution time 24+ hours]: TrainNotebook. ipynb
To regenerate the dynamics pictures of figure on a new network, please run: AnalysisOfTrained nmRNN ipynb
MLE_Fits
* cuvironments/Dockeet
* To reproduce the MLE fits and correlation analysis on a previously generated agent sequence please run: Notebooks Boookks/Pii_nnllll AaRRNN_Aeentt aeoent_Analyiss.... ipynb
Fig5
Multitasking
* Yang199uultitsskingghl
environmenn/Dockerfii
To train
network please Beno__Satiialleetoor--pppa ipynb To analyze
[execution
time
48+ hours]
a new network
please
run
ipynb
AdaptatibilityMeurrogo A repo not used in the paper to study the adaptability of the mmRNN. This repo is in development welcomes comments, corrections, and contributions.
Note: The code is provided at publication in its current most robust form. Further code revisions are underway to merge the numerous environments while avoiding dependency conflicts.
References
[1] Eve Marder, Sonal Kedia, and Ekaterina O. Morozova. "New insights from small rhythmic circuits". In: Current Opinion in Neurobiology 76 (Oct. 2022), p. 102610. ISSN: 0959-4388.
DO1: 10 1016 conb. 2022 102610 URL: https www sciencedirect. com/ (visited on 05/06/2025).
science/
[2] Cornelaa I. Bargmann. "Beyond the connectome: How neuromodula-
tors shape neural circuits".
In: BioEssays 34.6
(2012).
_eprint:
pp. 455- ISSN:
1521-1878. DO1: 10. 1000000000ee.// 201100185
URL: httpp:::. inw.neiibrarry......mmmmmaaaaaae wiley
com com/doi/abs/ 10. www0//0eeesseee 2aeee (visited on 05/06/2025).
[3] Yuhan Helena Liu et al. "Cell-type-speccne neuromodulation guides synaptic credit assignment
a spiking neural network". In: Proceedings of the National Academy of Sciences 118.1
(Dec. 2021). Publisher: Proceedings of the National Academy of Sciences, e211 18211181111 DOr:
10. N0.1073/pnzn pnas. mamas.2111822 2111821118 URL: Ihttps: :/opps://en http://wwwp....o..m/////. wttp:///o///oooo///eeee/e 1073/ ppaa//
1111821118| (visited on 05/06/2025).
[4] Yuhan Helena Liu al. "Biologically-plauiibbe backpropagation through arbitrary timespans en. In: Advances in Neural Information Processing Systems 35
via local neuromodulatone
(Dec. 2022). pp. 17528-17542 URL.: htt.hhttps//pprceedngg nwwrinu.....mm cttpps.cccppppeee wwtpwcpppppppp....pp files/
paper / 2022 htte://ahhhfffaaaffaee httaa///cccaaacooooooooo//////aa -bt.taaaaaaaaaaaaaabaaaaaa
Abstract Conference
html (visited on 05/06/2025).
45

---

[5] Özge D. Özgete, Aditi Banerjee, and Pascal S. Kaeser. "Mechanisms of neuromodulatory volume transmission". en. In: Molecular Psychiatry 29.11 (Nov. 2024). Publisher: Nature
Publishing Group. pp. 3680-3693, ISSN: 1476-5578, DOr: 10. (038/s41380 024-00008----8
3. URT.: https: www nature. com / artaaiitllee s41380 024 02608 3 (visited on 05/06/2025).
[6] Jinyung Hong and Theodore P. Pavlic. Learning Modulate Random Weights: Neuromodulatiooninnsiire Neural Networks For Efficient
Continual Learning.
arXiv:2204.04299 [cs]. Oct. 2023. DOT: 10 48550 arXiv 2204 04297
URL:
httt:://axivv org/ /2044. 2atp///200.0297777.7 (visited on 05/06/2025).
[7] Allison Cruikshank et al. "Dynamical questions in volume transmission". In: Journal of Biological Dynamics 17.1 (Dec. 2023). ISSN: 1751-3758.
DOI: 10. 10800 17580717 2023
2269986 (Visited on 05/08/2025).
[8] Rachel Grashow, Ted Brookings, and Eve Marder. "Reliable neuromodulation from circuits with variable underlying structure". In: Proceedings of the National Academy of Sciences 106.28 (July 2009). Publisher: Proceedings of the National Academy of Sciences, pp. 11742- 11746. DOI: 10. 1073/ 10as/pnasss..c/ 0905614106
URL: https::/wwxx wnws..pa...... org/ htgg/dii/10.110
pnas. www.090566144 4 (visited on 05/06/2025).
[9] Michael C. Avery and Jeffrey L. Krichmar. "Neuromodulatory Systems and Their Interac- tions: A Review of Models, Theories, and Experiments", English. In: Frontiers in Neural
Circuits 11 (Dec. 2017). Publisher: Frontiers. iSsN: 1662-5110 DOr: 10 3389 fncir
2017 00108 URL: htttps://ww..ftttt http://wwwwfrtttorriii..c wwwhttps: //www. frontiersin. org/
3389 fncir. 2017.00108/ffll/ (visited
05/06/2025).
[10] David A. McCormick, Dennis B. Nestvogel, and Biyu J. He. "Neuromodulation of Brain State and Behavior", en. In: Annual Review of Neuroscience 43. Volume 43, 2020 (July 2020). Publisher: Annual Reviews, pp. 391-415 ISSN: 0147-006X, 1545-4126. Dor: 10 1146/
annurev-neuro- 1et.--00011---0
annualreviewe www.e/wwwwww...e.ee.ee.ee. /xgg/ontet////// journals/10. / anwurev- 1000000000--00 105424 (visited on 05/06/2025).
URL:
[11] Guillaume Bellec al. "A solution to the learning dilemma for recurrent networks of spiking
neurons". en. In: Nature Communications 11.1 (July 2020). Publisher: Nature Publishing
Group, p. 3625. ISSN: 2041-1723, DoL 1038 s41467 020 17236-y. URL. https
wwww.nature.em com htt.c///aarccee///111444441..441 /ttp://41477-000 17236-y (visited 10/21/2024).
[12] James M Murray. "Local online learning in recurrent networks with random feedback". In:
eliife 8 (May 2019). Ed. by Peter Latham, Michael Frank, and Brian DePasquale. Publisher:
eLife Sciences Publications, Ltd, e43299. ISSN: 2050-084X. DOT: 10. 7554/ 7554/eLife eLiff.43299 URL: htt:httts://do// htt:///oooorg/1////// 7554//100.554//iifffee 43299 (visited on 10/21/2024).
[13] Thomas Miconi et al. "Backpropamine: training self-modifying neural networks with differ- entiable neuromodulated plasticity". en. In: Sept. 2018 URL: http:tt//////peeeeevies net/
forum?id=-11rAAAA
(visited on 05/06/2025).
[14] Sarthak Mittal al. "Explicit Knowledge Factorization Meets In- In-Context Leamning: What Do We Gain?" en. In: May 2024. URL: http:tt:///ppeeeeeiieeet net htt.://wwww...eeeeoououoddddddo.
(visited on 05/08/2025).
[15] Julia C. Costacurta
Structured flexibiiity in recurrent neural networks via neuromodulationn
en. Pages: 2024.07 077.666605315. Section: New Results. July 2024. DoL: 10. 1101/2021.11111 .7 26
www.ox/oorg//ooooooooeoeeooo wttpv///..r//ootee///000/ 11011/0..11111100 1101//024.
605315 URL. pattps:/wwwy bworxiv..rr............iieeo...t (visited on 10/21/2024).
26. 605315v1
[16] Kyle Aitken and Stefan Mihalas. "Neural population dynamies of computing with synaptic modulations". In: eLife 12 (Feb. 2023). Ed. by Gianluigi Mongillo, Joshua Gold, and Omri Barak. Publisher: eLife Sciences Publications, Ltd, e83035. ISSN: 2050-084X. DOI: 10. 7554/ $Life.83033 URL.: httt:://tss://oii.. org/10, 10. 75547755//// e0ife.e 83035 (visited on 05/06/2025).
[17] Ben Tsuda et al. Neuromodulators generate multiple context- behaviors in a recurrent neural network by shiftting activity hypertubes. en. Pages: 202 00.. 31.4446422222 Section: New
Results, Jan. 2022. DO1: 10. 1101/2022 05 31 446462
URL: https:/// ://www/ biorxiv
446462424222222 org/content.. 1101 35.3..4444422222 (visited on 0///5//0255.
05.
[18] David Ha, Andrew Dai, and Quoe Le. /lyperNetwwrkss. arXiv: 10w::1666900000 09100. Dec. 2016. DOI:
10. 48550/ aet00//rxiv.... 1609. 09106
URL: htll:tttp://arat org abs /elbs///1...... 09106
(visited on
10/21/2024).
46

---

[19] Johannes von Oswald et al. Continual learning with hypereetworks. arXiv: MeeX:vv:990000 1ww...9000009999 Apr. 2022. DOI: 10. 23550/arXivv 00695 (visited on 10/21/2024).
URL: http:/htp://aaa0 www./bs///a/o/oooo//oo//oee
abs/ https/1900.006090
[20] Johannes von Oswald al. Transformers learn in-context by gradient descent. arXiv:2212.07667. May 2023. DO1: 10. 00..48550/rr 20500//rXiv.... 2212.07677
URL: htt.:httpp://tttieee
org/ 2tt/ab//2222.0075/ (visited 10/21/2024).
[21] Imanol Schlag, Kazuki Iric, and Jürgen Schmidhuber. Linear Transformers Are Secretly Fast June 2021. DOI: 10. 48544/55////// http://xviivvii.. 11174
Weight Programmerss. arXiv: 1...i:22222.11111.
URL: http:://axiiii org/
wwwggaa//// abs 2202 11174 (visited on 10/21/2024).
[22] Albert Gu et al. "HiPPO: Recurrent Memory with Optimal Polynomial Projections". In: Ad- vances in Neural Information Processing Systems. Vol. 33. Curran Associates, Inc., 2020, pp. 1474-1487. URL: https
proceedings 002f0bb6efb3a6 0B220066eeb336611223376666666 p neurips cc / paper 2020 hash / Abstract html (visited on 05/06/2025).
[23] Benjamin Ehret al. Continual Learning in Recurrent Neural Networks, arXiv:2006.121009
Mar. 2021. DO1: 00111..4400/11 48550/arXiv 2006. 12109
URL: http: /tttp://arxiin /
12109 (visited on 10/21/2024).
[24]
Arthur Pellegrino, N. Alex Cayco Gajic, and Angus Chadwick. "Low Tensor Rank Learning of Neural Dynamics". en. In: Nov. 2023. URL:
http://wwwwweeetttutreeeeo ner/forum idd
WcoX8eJJjI (visited on 05/08/2025).
[25] Marcin Andrychowicz et al. Learning to learn by gradient descent by gradient descent. arXiv:1606.04474 Nov. 2016. DOT: 10. 00.4445555rraa http//..vvvv......... 11116666644444
URL: http: //arxiv
org/abs/1606. 04474 (visited on 10/21/2024).
[26]
Jane X. Wang et al. Learning to reinforcement learn.
[cs]. Jan. 2017. DOT: 05763 (visited on
10. 41.45500arXiv.... 1611. 05763
URT.:
http:///xxvvvrrrge
05/15/2025).
[27] Guangyu Robert Yang al. "Task representations in neural networks trained to perform many cogmitive tasks". eng. In: Nature Neuroscience 22.2 (Feb. 2019), pp. 297-306. ISSN: 1546-1726. DOT: 10. 1000///419930000 018-93---8-000 018--00000--31--22
[28] Laura N. Driscoll, Krishna Shenoy, and David Sussillo. "Flexible multitask computation in recurrett networks utilizes shared dynamical motifs". en. In: Nature Neuroscience 27.7 (July 2024). Publisher: Nature Publishing Group, pp. 1363. ISSN: 1546-1726. DOI:
10. 024-41199-2---100 020-00--22----88--8-111-88
URL: htll::htps:: //ts:///www nature com wwwraiieeeeeee111
024-01668-6 6 (visited on 10/21/2024).
[29] Anthony Zador et al. "Catalyzing next-generation Artificial Intelligence through NeuroAI". en.
In: Nature Communications 14.1 (Mar. 2023). Publisher: Nature Publishing Group. p. 1597. ISSN: 2041-1723. Dor: 10. 1038 / #4146744677777 023-3323333330003 37 177188- URL: https: htmp:://wwwww
///www nature
0ttp://411477-0033331--------- 37180-x (visited on 05/15/2025).
[30] Jonathan Cornford al. "Learning live with Dale's principle: ANNs with separate excitatory
and inhibitory units". en. In: Oct. 2020. URI.: (htpps
openreview. / net bet/forum forum ? idd
eU776ZYxEpz (visited on 05/15/2025).
[31] Casey M. Schneider-Mizell et al. "Inhibitory specificity from a connectomic census of mouse visual cortex". en. In: Nature 640.8058 (Apr. 2025). Publisher: Nature Publishing Group. pp. 448-458. ISSN: 1476-4687, DOL: 10. 1038 / s41586 024 - 07780 8 URL: hhtps 01124-07780-2 (visited 05/15/2025).
//www.nature com
[32] Mikail Khona et al. "Winning the Lottery With Neural Connectivity Constraints: Faster Learning Across Cognitive Tasks With Spatially Constrained Sparse RNNs". In: Neural
Computation 35. 11 (Oct. 2023), pp. 1850-1869 ISSN: 0899-7667. DOI: 10. 1162/neco__ce
(visited on 05/06/2025). [33] Jascha Achterberg et al. "Spatially embedded recurrent neural networks reveal widespread links between structural and functional neuroscience findings". en. In: Nature Machine Intelligence
01613 URL: https: //doi. org 10. www11//////oa_0011
5.12 (Dec. 2023). Publisher: Nature Publishing Group. pp. 1369 1381. ISSN: 2522-58232 5839. DOI:
10.1038/ s422562226666 -2000033-00000 00748-9
URL: httpp::/wwwt nature. com/ articcessee2226ee
023 023-00748-9 9 (visited on 05/15/2025).
[34] Jonathan Schwarz al. "Powerpropagation: A sparsity inducing weight reparamcterisationn en. In: Nov. 2021. URL:
net/forum? wiwpummmddddaaasxppxm... (visited on
05/15/2025).
47

---

[35] Salvatore Torquato. "Hyperuniform States of Matter". In: Physics Reports 745 (June 2018). arXiv:1801. 0092449944444 [cond-mat], pp. 1-95. ISSN: 03701573, DO1: 10. 1016/j physrep. 2018
03.001 URT.: http:://axiiiii org/ htt///ass/8000 1801 06924 (visited on 05/15/2025..
[36] Kamesh Krishnamurthy. Tankut Can, and David J. Schwab. "Theory of Gating in Recurrent Neural Networks". In: Physical Review 12.1 (Jan. 2022). Publisher: American Physical
Society. p. 01101111111111111111111111110011 DOL: 10 1103 www.00//hyyyee..e 12. 011011 URL: htttps://linkk aps org/
doi/10. 10w..111000//hyee..... 12. 011011 (visited on 05/16/2025).
[37] Blake A. Richards et al. "A deep learning framework for neuroscience". en. In: Nature Neuroscience 22.11 (Nov. 2019). Publisher: Nature Publishing Group, pp. 1761-1770. ISSN:
1546-1726 DOI: 10 1038 / s41593 019 0520 2 URL: https: www nature. com
srricles/s41593 019 0620- 2 (visited on 05/08/2025).
[38] T. Konstantin Rusch and Siddhartha Mishra. "Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies", en. In: Oct. 2020. URL: https:::/s http://///eeeeeeeeeeeeeee net www.:neeeooounnne--33333333. 05/09/2025).
(visited on
[39] Shuhei Watanabe. Sree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance. arXivv [es]. May 2023, DOI:
10. 48550/50//5 arXiv 2304. 11127 05/09/2025).
URL: ltttp://aartti org/ aba/2304. 11127 (visited on
[40] Wolfram Schultz. "Predictive reward signal of dopamine neurons". In: Journal of neurophysi- ology 80.1 (1998), pp. 1-27.
[41] Bilal A Bari et al. "Stable representations of decision variables for flexible behavior". In: Neuron 103.5 (2019), pp. 922-933.
[42] John Schulman et al. Proximal Policy Optimization Algorithums, www.::...........001211 [cs]. Aug. 06347
2017. DOT: 10. (visited on 05/16/2025).
1707. 06347
URT.: htt.::tt:///axi..orgee.e..e
[43] Jane X. Wang et al. "Prefrontal cortex as a meta--riinforeement learning system". en. In: Nature Neuroxcience 21.6 (June 2018). Publisher: Nature Publishing Group, pp. 860-868.
ISSN: 1546-1726. DOL: 10.
URL: wntp:htpp////. //www. nature. com/
articles/ss41593-01888 018-- (
(visited on 05/15/2025
[44] David L. Barack and John W. Krakauer. "Two views on the cognitive brain".. en. In: Nature
Reviews Neuroscience 22.6 (June 2021). Publisher: Nature Publishing Group, pp. 359-371
ISSN: 1471-0048 DOL: 10. 0000///41588 021 0044004488808 6 URL: https: // /wwww nature
com/ 0aartiicees//41153000111---00 (visite on 05/15/2025).
[45]
David Meunier, Renaud I Lambiotte, and Edward T. Bullmore. "Modular and Hierarchically
Modular Organization of Brain Networks". English. In: Frontiers in Neuroscience 4 (Dec.
2010). Publisher: Frontiers. ISSN: 1662-453X. DOI: www93//ffniii 2010 00200, URL: https: /ww. frontiersin org/ journals
httpraaallneeurrieennneee neuroscience aatticlss 10 3389 /
fnins. 20ins.2010..m 00200/ful] ( 05/15/2025).
[46] Zizhen Yao et al. "A high-resolution transcriptomic and spatial atlas of cell types in the
whole mouse brain".. en. In: Nature 624.7991 (Dec. 2023). Publisher: Nature Publishing Group, pp. 317-332 ISSN: 1476-4687. DOI: 10 1038 s41586 023 06812 URL.: 00023-06681222 (visited on 05/16/2025)
https://www nature. com htt/aaaaiceeeeeaaaaa
[47] Thomas Miconi and Kenneth Kay. "An active neural mechanism for relational learning and
fast knowledge reassembly". eng. In: bioRxiv: The Preprint Server for Biology (Sept. 2023),
p. 2023.07.27.5507733 ISSN: 2692-8205. DOI: 10. 1101/2023.0.. 07 27 5500739
[48]
Scott W. Linderman et Recurrent switching linear dynamical systems. Oct 26, 2016. DOI:
10. 485500 118550/arXiv... 1610. 08466 arXiv: 16100 [s4at[
URL: httt:://rxxiv.or org/
abs/1610. 08466 (visited on 10/24/2025).
[49] Steven L. Brunton et al. Modern Koopman Theory for Dynamical Systems. arXv2200226..
Oct. 2021. DOI: 10. 48550/ arXiv 2102. 12086
URL: /ttt:://arxiinn org/ abs/2102
12086 (visited on 10/21/2024).
[50] Michael T. Pearce et al. "Bilinear MLPs enable weight-based mechanistic mechaanss interpretabilliyy""" en. In: Oct. 2024. URL: attps:///on http://pwweeeeeeeeeeeeeorrrrraaa 05/08/2025).
(visited on
48


Reviews:
[{"review_id": "QOgzs3Mn6r", "reviewer": "Reviewer_gAEg", "summary": "The authors present a context factorization algorithm which is inspired by the roles of neuromodulators in the brain to alter circuit dynamics and act as a third factor in Hebbian learning. They incorporate neuromodulators, which have their own dynamics, as an additional factor modulating recurrent connectivity in an RNN, as well as several different classes of decoders. They show that the RNN uses the neuromodulators to approximate gradient descent in-weights, and can generalize compositionally. They then demonstrate its efficacy on RL tasks and that it leads to brain-like solutions in multi-task learning when using a spatial embedding.", "questions": "1. How does an e-nmRNN differ from an nmRNN (in words and equations)? What is the particular advantage of the \"endogenous\" part?\n\n2. Are there situations in which you would expect the algorithm to fail?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n*Quality:*\n- The paper contains a large number of carefully-conducted analyses and simulation settings from supervised learning to RL\n- The algorithm learns to solve meta-learning tasks significantly faster than the baselines\n- The paper demonstrates a strong mix of theory and simulation, illustrated for instance by the analysis of the emergent gradient-descent-like updates in the RNN\n- The paper cites widely and is well-grounded in neuroscience\n\n*Clarity:*\n- The diagrams in the figure help describe the concepts well; I appreciated the text labels to guide the reader\n\n*Significance:*\n- The paper represents a potentially significant advance for machine learning by providing theoretical analyses and simulations demonstrating the utility of multiplicative gating in RNNs, and for neuroscience by using this as a model for neuromodulators\n- It recovers known features of neuroscience such as emerging RPEs\n- The authors detail plans for follow-up projects in neuroscience\n\n*Originality:*\n- As far as I know, the machine learning demonstration of the utility of neuromodulator-like factors for continual learning is novel\n\n**Weaknesses:**\n\n*Quality:*\n\n*Clarity:*\n- Not all variables are defined when equations are presented (although they are later in the list, lines 77-86). Perhaps it is possible to guide the reader better here?\n- The paper requires a significant amount of prior knowledge in neuroscience in order to be easily understood (I would not expect readers to know Dale's law without definition, for instance)\n- nmRNNs are frequently referenced in the paper; it would be helpful to briefly explain how they differ from the e-nmRNNs here\n\n*Significance:*\n-  It would be good to emphasize the paper's original contributions a bit more, perhaps in a bulleted \"contributions\" list in the introduction, to make clear what its additions are to prior work\n\n*Originality:*\n- As the authors acknowledge, representing neuromodulators as gating is not a new idea in NeuroAI", "was_revised": true, "final_justification": "This paper presents a good implementation of an interesting computational idea, and presents a large number of analyses. Although I understand other reviewers' concerns that the limits of the biological plausibility of the model need to be more carefully described, the model remains worthy of publication, and I am confident the authors will be able to make those changes for the camera-ready version.", "rebuttal": "We sincerely thank Reviewer gAEg for your positive assessment and for recognizing the value of our work. We are particularly grateful for your comments that the paper contains a \"large number of carefully-conducted analyses,\" demonstrates a \"strong mix of theory and simulation,\" and represents a \"potentially significant advance for machine learning... and for neuroscience.\"\n\nPer your suggestion to make our contributions clearer, we offer this bulleted list which we will endeavor to compress into any revised manuscript:\n\n* A Unifying Computational Principle: We propose \"context factorization\" as a core computational principle implemented by neuromodulation, unifying its dual roles in altering circuit dynamics and gating synaptic plasticity.\n* A Bio-Inspired Network Architecture (e-nmRNN): We introduce the endogenously neuromodulated RNN (e-nmRNN), a type of context-factorized hypernetwork derived from biophysical principles, where the network learns to generate its own modulatory context.\n* A Mechanism for Online Credit Assignment: We demonstrate that the e-nmRNN architecture naturally enables targeted online credit assignment. Its learning rules are gated by the neuromodulatory state, allowing the network's dynamics to approximate gradient descent and facilitate rapid, in-context meta-learning.\n* Strong Compositional Generalization: We show empirically that the e-nmRNN achieves respectable performance on compositional generalization tasks, outperforming and matching several strong baselines.\n* Emergent Biologically aligned phenomena: We find that when trained on complex multitasking benchmarks, the model develops emergent properties that mirror neurobiology, such as modularity, cell-type specialization, and distinct functional timescales, enhancing its interpretability.\n* Application to Reinforcement Learning: We demonstrate that the e-nmRNN can learn to encode critical learning signals, like Reward Prediction Error (RPE), within its neuromodulator dynamics, providing a mechanism for RPE-gated online adaptation in RL environments.\n\nWe now address your specific questions below.\n\n**Q1: How does an e-nmRNN differ from an nmRNN (in words and equations)? What is the particular advantage of the \"endogenous\" part?**\n\nThis is an excellent question that gets to the heart of our model. The \"endogenous\" nature of our e-nmRNN is its defining feature: the neuromodulatory state $\\vec{n}(t)$ is generated by the network's own recurrent activity $\\vec{r}(t)$ through a learned projection matrix $R$. The dynamics are coupled:\n$$\n\\tau_r \\dot{\\vec{r}} = -\\vec{r} + \\phi\\left( (W + \\sum_k n_k T_k) \\vec{r} + U\\vec{I} \\right)\n$$\n\n$$\n\\tau_n \\dot{\\vec{n}} = -\\vec{n} + \\theta(R\\vec{r} + Z\\vec{n})\n$$\n\nThis creates a closed-loop, self-modulating system. This contrasts with other models where the modulatory signal might be an external input (e.g., a task-ID vector). The advantage of the endogenous approach is that it allows the network to learn to generate its own internal context states directly from its neural population dynamics. This enables it to autonomously infer the current context from the input stream and adapt accordingly, which is crucial for the meta-learning and RL results we demonstrate, where explicit context signals are not always available.\n\n**Q2: Are there situations in which you would expect the algorithm to fail?**\n\nYes, thank you for this insightful question. We see two primary failure modes:\n* Data-Limited Overfitting: The model's rich, multiplicative dynamics make it more powerful but also more prone to overfitting than simpler RNNs when training data is scarce. This can be seen in our original Fig. 3 at high task complexity ($K > 15$), where the performance of the e-nmRNN degrades more rapidly than the GRU. In response to your inquiry, we have also performed an analysis of the generalization gap, which shows this phenomenon more clearly as the e-nmRNN’s gap between training and validation loss grows more rapidly than that of the equivalent GRU. A future research topic outside the scope of this work will investigate the effective forms of regularization to avoid this trap. \n* Extremely Rapid Context Switching: The neuromodulator dynamics $\\vec{n}(t)$ have intrinsic, learned timescales ($\\tau_n$). If a task required context switches much faster than these timescales, the modulatory state would be unable to keep pace, leading to performance degradation. This highlights a fundamental trade-off between maintaining a stable context representation and achieving rapid adaptability.\n\nWe appreciate the reviewers notes on places where clarity can be improved and take these comments seriously. In future revisions, we will pay special attention to undefined variables, and the use of specialized neuroscience nomenclature without definition (e.g. Dale's law)."}, {"review_id": "R863SeFytR", "reviewer": "Reviewer_yq59", "summary": "This paper proposes a biologically inspired recurrent neural network, e-nmRNN, motivated by the central role of neuromodulation and volume transmission in the brain. The model operationalizes the concept of context factorization, wherein neuromodulator concentrations dynamically gate network connectivity, providing an interpretable mechanism that links biological mechanisms and computational principles. Through both analytical derivation and empirical evaluation, the paper demonstrates that context factorization enables targeted online credit assignment (with learning updates gated by the neuromodulatory context) and supports compositional generalization. The work includes experiments in supervised learning, meta-learning, and reinforcement learning, showing superior or competitive performance to standard baselines, as well as emergent modularity and cell-type specialization mirroring biological observations.", "questions": "1. The spatially embedded variant outperforms the standard e-nmRNN. Could this improvement be attributed to effective sparsity induced by spatial constraints? \n\n2. In the multi-task decision-making experiments (section 6), does contextual differentiation emerge autonomously from the model dynamics, or is context explicitly provided as input? \n\n3. What precisely differentiates the modeled “dopamine” and “serotonin” neurons in the architecture? Is it solely based on opposite modulation slopes (e.g., positive vs. negative sensitivity) ?", "limitations": "see weaknesses and questions.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "***Strengths***:\n\n1. The paper is clearly written and easy to follow.\n\n2. The proposed endogenously neuromodulated RNN (e-nmRNN) presents a novel architectural idea. The work systematically relates this model to diverse learning settings and biological phenomena, offering valuable insights into the functional role of neuromodulators in the brain.\n\n3. Experimental results are well-supported by appropriate baselines, and related literature is thoroughly discussed, situating the work well within existing research.\n\n\n\n***Weaknesses***:\n\nTransformers and architectures like Hippo-LegT derive much of their power from strong scalability and parallelization capabilities, which enable them to handle long sequences and large datasets efficiently. In contrast, the scalability properties of e-nmRNN are less clear. It would be valuable for the paper to discuss the computational cost and parallelizability of e-nmRNN, especially in comparison to modern scalable architectures.", "was_revised": true, "final_justification": "I have no major concerns with this work and believe it's a good contribution to the field. I support its acceptance.", "rebuttal": "We thank reviewer yq59 for your positive review. We are delighted that you found the paper \"clearly written and easy to follow,\" that our proposed model \"presents a novel architectural idea,\" and that our \"experimental results are well-supported by appropriate baselines.\" We address your insightful questions and comments below.\n\n**Weakness: Scalability properties of e-nmRNN are less clear... discuss the computational cost and parallelizability...**\n\nThis is an important point. As a recurrent architecture, the e-nmRNN is inherently sequential during inference, which makes it less parallelizable than models like Transformers. This represents a trade-off: we gain powerful inductive biases for online, continuous adaptation within a single data stream at the cost of the massive parallelizability that makes Transformers so scalable for large-batch offline processing. We position the e-nmRNN as an architecture well-suited for applications like real-time control, agent-based learning, and brain-computer interfaces, where online processing is paramount. We believe its value also stems from its connection to neurobiology, providing an interpretable model for scientific inquiry even if it is not intended to compete with Transformers on large-scale sequence processing benchmarks. \n\n**Q1: The spatially embedded variant outperforms the standard e-nmRNN. Could this improvement be attributed to effective sparsity induced by spatial constraints?**\n\nYes, we agree and believe this is a primary reason for the improved performance. The spatial embedding acts as a strong and biologically-inspired regularizer. It simultaneously imposes sparsity and encourages the formation of local processing motifs by making connection strength decay with distance. This regularization appears to prevent overfitting and guide the learning toward more robust and generalizable solutions, particularly on the complex multitasking benchmark.\n\n**Q2: In the multi-task decision-making experiments (section 6), does contextual differentiation emerge autonomously from the model dynamics, or is context explicitly provided as input?**\n\nThe network is provided a one-hot task ID at the beginning of each trial. However, the crucial finding is how the network learns to use and maintain this information. The architecture's inductive bias encourages this context to be encoded and dynamically maintained within the neuromodulatory state $\\vec{n}(t)$. As shown in Figure 5C, a linear decoder trained on these neuromodulator concentrations can reliably identify the current task long after the initial cue has passed, whereas a decoder trained on the neural activity $\\vec{r}(t)$ cannot. This demonstrates that the e-nmRNN learns to represent the task context in its endogenous modulatory state, using it to shape the circuit dynamics appropriately for the task at hand.\n\n**Q3: What precisely differentiates the modeled \"dopamine\" and \"serotonin\" neurons in the architecture?**\n\nIn our model, the differentiation is functional and learned, not hardcoded. We use names of known neuromodulators for illustrative purposes. A unit becomes a \"dopamine-releasing neuron\" (a term we would clarify to mean \"a unit whose activity promotes dopamine release\") if the corresponding column in the learned release matrix $R$ becomes selective for that modulator. For instance, if the $j$-th column of $R$, $R_{:j}$, becomes approximately a one-hot vector $[1, 0, 0, ...]^T$, then neuron $j$'s activity $r_j$ primarily drives the dynamics of the first neuromodulator, $n_1$. The functional effects of \"dopamine\" vs. \"serotonin\" are then determined by how their respective rows in the learned tensor $T$ gate synaptic transmission throughout the network to solve the given tasks. Here, we primarily use this description as an evocative example of how interpretation could proceed, but intend to reduce the emphasis in future revisions. We thank the reviewer for calling this to our attention."}, {"review_id": "BeFZvvIiR4", "reviewer": "Reviewer_8PJy", "summary": "In this paper, the authors consider a model of a recurrent neural network enhanced with a simplistic version of volume transmission mediated by neuromodulators. They show that these hybrid networks can be trained to solve certain representative tasks that involve aspects of online credit assignment, compositional generalization and a reinforcement learning task.", "questions": "Define “context factorization”: this is the “central claim” of the paper but this concept is not well explained, so we do not understand what difficult or biologically relevant feature this model is achieving (that previous models did not). \n\nSection 2.2 “Multiplicative gating of dynamics” is unclear. It should either be expanded to properly explain the two distinct bifurcation mechanisms, or cut altogether to focus on explaining better the results in subsequent sections.  The discussion has a more precise statement about this than the results section.  \n\nFigure 3: the results are not very impressive, Oscillatory RNNs and gated recurrent units overperform. Comment on this aspect, and please explain why some models overperform. \no\tIs this the “overfitting” issue? Also please explain this issue better, identify which parts of which figures show overfitting, and propose concrete solutions – model reduction, bigger training sets?  \n\t\nFigure 4: test a bandit where the correct arm switches more unpredictably; here, especially with the anticipatory switching, it just looks like the RNN happens to oscillate at the “right” frequency to match the reward switching frequency, without tracking it. \no\tMaybe repeat for increasingly stochastic switching (vary both the switching frequency and the inter-switch duration variability) and plot performance versus these stochastic switching parameters?\n\nFigure 5: it would be interesting to plot the spatial localization of neuromodulator release. Make a plot of cells at their 2D positions and color them according to their average or instantaneous release of a neuromodulator. \no\t Relatedly: do cells of the same type colocalize to different neighborhoods (thus concentrating their range-limited – eq. in panel A –  synaptic interactions within their subgroup) or are they spread out to interact homogeneously with other cell types? \n\nFigure 5: poor resolution, heatmaps of panel E have meaningless ticks.", "limitations": "The limitations section is currently brief and non-specific -- this could benefit from more precise statements, perhaps commenting on specific modeling assumptions or the set of tasks considered in this work. For instance, the well-mixed approximation, the various steady-state approximations from the LIF model to the trainable e-nmRNN, where all interactions are tensor products (non-linear dynamics though arise from multiplying n(t)r(t)) inside a global nonlinearity could be discussed. Also, please propose more concrete solutions for the overfitting problem stated here.", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: The paper considers an often overlooked aspect of neural network function, and is a step towards a deeper understanding of how volume transmission when combined with classical synaptic communication leads to novel and more efficient forms of neural computation. \n\nWeaknesses: The paper is incomplete in its current form, and significant revisions (well beyond the scope of this submission) are necessary to bring it to par. \n\nFirst, the main text contains equations where a significant number of the variables, parameters and abbreviations are not defined. One should not have to read the appendix to know what a variable represents. To give a few examples (which is certainly not an exhaustive list), \n1. LIF is not defined in line 73\n2. In the equation at the end of page 2, S is not defined. \n3. In the equations above line 74, the nonlinearities \\phi and \\theta are not defined. In the Appendix, different symbols are used to represent these quantities, and that makes it further confusing. \n4. In the equation above 87, D is not defined (and later D is also used for dimensionality)\n5. In the equation above line 121, A and \\epsilon are not defined. \n\nSecond, the Appendix has trailing sentences that are left incomplete -- for example, lines 43 and 44. It is also poorly organized -- the derivation of the final model from the LIF model is hard to follow (e.g. unnecessary one-sentence subsections, the function phi appears several times before being derived, etc.)\n\nThird, the main text has several grandiose statements about the framework addressing important biological questions rather than critical details on the tasks and training of the models. Currently, the main text is unspecific to the point where it is unclear what is being done with this model. A few examples:\nFigure 5: Unclear what the 20 tasks are and how the model is trained on all of them. \nFigure 3: The distinction between pre-training (trained on a fixed training set of sine curve parameters?) and in-context learning (new combinations of sine curve parameters not in the training set?) is unclear.\nIn Figure 4, the switching of the bandit arm reward probabilities seems almost periodic. It is unclear from the main text/Appendix what parameters are being used. \n\nFourth, the model proposed by the authors involves unrealistic biophysical assumptions. For example, a key assumption in the model is that neuromodulators are well-mixed (i.e., that the mixing time is negligible compared to the timescale of neural computation), but it is unclear how this is physically plausible. Neural activity operates on timescales of a few tens of milliseconds, while diffusion of neuromodulators occurs at much slower timescales. The authors also assume a quadratic \"chemical\" interaction between neuromodulators (last term in eq. 18) which is not justified. What is the biophysical basis behind this interaction term? The authors also add in nonlinearities to the equations (lines 115 and 127 of the Appendix) without a clear justification, and claim that this is a \"modeling assumption\". This is OK if the paper also didn't claim to be biologically realistic (lines 195 to 197 of the Appendix).", "was_revised": true, "final_justification": "I cannot comment on most of the rebuttal until the paper is revised. However, my point that the paper was not in a publishable form when first submitted still stands. As it seems like the paper will likely be accepted, I encourage the authors to significantly revise the paper for clarity and to make clear what central claims are being made. I will retain my original score. \n\nHere is my response to some of the other comments:\n\n\"Well-Mixed\" Assumption: We agree this is a simplification. We interpret it as a timescale separation assumption, where diffuse neuromodulator concentrations establish a stable \"context\" for fast neural dynamics ($\\tau_n \\gg \\tau_r$). While diffusion itself is slow, the establishment of a new ambient concentration across a local circuit can occur on a timescale relevant for trial blocks in cognitive tasks. We feel this is supported by two observations: 1) The integrative nature of the neuromodulator concentration acts as a low-pass filter on the population dynamics, which in turn modifies the mean population dynamics. This is a tool for achieving rich multi-timescale dynamics. 2) The neurobiological observation that single neuromodulator-releasing neurons project widely across the cortex suggests that biology could accelerate the approach to a well-mixed state by increasing the density of release loci.\n\nI agree that a new ambient concentration can likely be established on timescale of trial blocks, but this is not the assumption made in the paper. The possibility of \"rich multi-timescale dynamics\" does not justify an assumption about biological timescales. Widespread projections of neuromodulatory neurons does not imply that the timescale of equilibration is fast. Moreover, the model explicitly considers endogenous neuromodulatory interactions.\n\nNonlinearities: These are standard in rate-based models to ensure stability and represent the saturating, positive firing rates of neurons. These nonlinearities also provide important numerical stability during training across a much wider range of hyperparameters.\n\nIf certain key assumptions were made for numerical stability rather than biological plausibility, it would help if the authors are more explicit that their model is not a biophysical model of a neuromodulatory NN but rather a biology-inspired model for recurrent computation. In the paper's current form, the authors draw inferences about what role volume transmission may play in biological contexts from their analysis.", "rebuttal": "We thank reviewer 8PJy for your thorough and critical review. We sincerely apologize that a lack of clarity in our initial submission led to a frustrating reading experience. We acknowledge, and we appreciate your efforts to identify these specific issues with such helpful granularity. We believe the concerns raised can be fully addressed through clarification, and your feedback will be invaluable in improving the manuscript. We have worked to address your points below and are confident that a revision would be substantially clearer and stronger.\n\n**On Clarity (Weaknesses 1, 2, 3):**\n\nWe are grateful for the detailed list of clarity issues. In a revision, we would ensure every variable is defined upon its first appearance and restructure the appendix. \n\nWe address your specific examples:\n1) Undefined variables: You are correct. In a revision, we will define:\n* LIF: Leaky Integrate-and-Fire neuron model, the starting point of our derivation.\n* $S_j(t)$: The spike train of the $j$-th neuron in the biophysical model.\n* $\\phi, \\theta$: The nonlinear activation functions for the rate ($\\phi$) and neuromodulator ($\\theta$) dynamics. We use functions like tanh or ReLU that enforce positive-only firing rates and saturation, consistent with the closed-form solution of the steady-state firing rate for a LIF neuron with a finite refractory period.\n* $D$: The linear readout matrix that maps neural activity $\\vec{r}(t)$ to the output $y(t)$.\n* $A_i(t)$, $\\epsilon_i(t)$: The pre-activation input or \"synaptic drive\" ($A_i(t)$) to neuron $i$'s nonlinearity, and the instantaneous error signal ($\\epsilon_i(t) = y_i(t) - y_i^*(t)$).\n\n2) Appendix Organization: We agree the appendix should be restructured for a clearer narrative flow. We take your comments here seriously.\n\nTask Details: We will expand all task descriptions.\n* **Multitask (Fig 5)**: The 20 tasks are from the well-established benchmark by Yang et al. (2019) and cover a range of cognitive functions. We will explicitly state this and cite the source more clearly. In the supplement, we will add a section dedicated to building the reader's intuition about the task set.\n\n* **Bandit Task (Fig 4)**: Your observation about the periodic switching is valuable. To address this potential confound, we ran a new experiment where the high-reward arm switched at stochastic, unpredictable intervals. The agent adapted robustly, confirming it learns a genuine model of the task state and is not merely entraining to a fixed frequency.\n\n**On Biological Plausibility (Weakness 4):**\n\nOur goal is a computationally powerful model that is inspired by biology, not a perfectly literal biophysical replica. We view our choices as principled abstractions designed to isolate core computational ideas.\n\n* **\"Well-Mixed\" Assumption**: We agree this is a simplification. We interpret it as a timescale separation assumption, where diffuse neuromodulator concentrations establish a stable \"context\" for fast neural dynamics ($\\tau_n \\gg \\tau_r$). While diffusion itself is slow, the establishment of a new ambient concentration across a local circuit can occur on a timescale relevant for trial blocks in cognitive tasks. We feel this is supported by two observations: 1) The integrative nature of the neuromodulator concentration acts as a low-pass filter on the population dynamics, which in turn modifies the mean population dynamics. This is a tool for achieving rich multi-timescale dynamics. 2) The neurobiological observation that single neuromodulator-releasing neurons project widely across the cortex suggests that biology could accelerate the approach to a well-mixed state by increasing the density of release loci.\n* **Interaction Term ($Z$ matrix)**: This term models effective interactions, not literal chemical reactions. It allows for one neuromodulatory system to influence another (e.g., release of one NM promoting or inhibiting the release/clearance of another), a known biological phenomenon (Morón JA, 2002, J Neurosci.). In practice, we often find this matrix is learned as approximately diagonal, simplifying to self-decay terms.\n\n* **Nonlinearities**: These are standard in rate-based models to ensure stability and represent the saturating, positive firing rates of neurons. These nonlinearities also provide important numerical stability during training across a much wider range of hyperparameters.\n\n**Specific Questions:**\n\n* Define \"Context Factorization\": We define this as the decomposition of a network's effective connectivity into a fast-varying neural activity and a slow-varying modulatory state (the 'context'). Mathematically, the effective synaptic weight from neuron $j$ to $i$ is factorized as:\n$$ W_{ij}^{\\text{eff}}(t) = W_{ij}^{0} + \\sum_{k=1}^{M} \\underbrace{T_{ijk}}_{\\text{\"dynamics\"}} \\underbrace{n_k(t)}_{\\text{\"context\"}} $$\nThis allows the low-dimensional context vector $\\vec{n}(t)$ to dynamically reconfigure the high-dimensional effective connectivity of the network. We use the word \"factorization\" to evoke the mathematical method of decomposing a function into a product of components—here, the components are \"context\" (encoded in neuromodulator concentrations) and \"dynamics\" (encoded in the resulting synaptic connectivity patterns).\n\n* Figure 3 Performance: Your observation is correct. The e-nmRNN's higher capacity makes it more prone to overfitting on this smaller dataset compared to a GRU. Our model's core strength is not necessarily to outperform all baselines on all tasks, but to provide a flexible and interpretable context-gating mechanism, the value of which is more clearly demonstrated in the multitasking and RL benchmarks.\n\n* Figure 4 Bandit: In response to your comment, we have trained additional e-nmRNNs on the bandit task with both more stochastic block switching and taking the trained agents (without learning) into new switching periods. In the first set of experiments, we observed  that the relative probability of the arms was detectable from the neuromodulator vector though admittedly with a less cyclic embedded trajectory. In the second set of experiments with fixed agents trained on oscillatory environments, we found that the e-nmRNNs where able to adapt to both longer switching periods and shorter switching periods. Given these quick turn around experiments, we expect that the more randomly switched bandits would indeed follow the reviewer's implicit hypothesis that they would more robustly learn online credit assignment that would allow the networks to be more robust to task variations. \n\n* Figure 5 Spatial Plot: Thank you for this excellent suggestion. We performed the analysis you proposed on the trained multitasking network. We found that units specializing in releasing a specific neuromodulator form notable spatial clusters, providing new evidence strengthening our claims of emergent, biologically-reminiscent organization.\n\nWe hope these detailed clarifications and new analyses address your concerns and further the value of our work. We are committed to using your thought-provoking feedback to significantly improve the manuscript."}, {"review_id": "xQdbkQdq03", "reviewer": "Reviewer_n2dL", "summary": "In this study, the authors present an ‘endogenously neurmodulated recurrent neural network’ (e-nmRNN), which uses neuromodulatory volume transmission to effectively induce hypernetwork-like dynamics in recurrent neural networks. The gating induced by the authors’ postulated neuromodulators enables in-context adaptation and compositional generalization on a variety of tasks, because gating suppresses unused dynamical motifs and prevents learning (ameliorating issues caused by catastrophic forgetting).", "questions": "Major concerns:\n\n1.\tHow does temporal credit assignment work in the learning algorithm employed here? The approach taken seems to be temporally greedy (appendix section 3) and is justified in terms of truncation of the gradient signal to ‘leading order’. However, temporally greedy approaches are in general not a well-justified approximation, and full temporal credit assignment using the backpropagation through time algorithm is required for good performance under most conditions without additional tricks. The e-prop and RFLO algorithms are cited, but these methods are not fully temporally local (they discard cross-neuron temporal interactions but not within-neuron temporal interactions), and the relationship between these methods and the present learning algorithm are unclear. As such, appendix section 3 would benefit from a rewrite emphasizing clarity and the present algorithm’s relationship to previous methods.\n\n2.\tTaking a step back, why were neuromodulators selected as the source of synaptic gain modulation underlying this method? For instance, very similar effects have been achieved via a model of thalamocortical gating (Logiaco et al. 2021), wherein thalamic input modifies the gain of some neurons (or could potentially excite or inhibit select portions of a dendritic arbor). The trouble with neuromodulators is that there are only a handful of distinct types: in this study it seems as though four were selected for most experiments. In contrast, for many types of learning, one could imagine that the relevant compositional primitives could range from tens to hundreds. How could variations in neuromodulatory tone in such a low-dimensional space function to adequately support learning of such a high diversity of primitives? And relatedly, is there any way of experimentally testing that this really is the function that neuromodulators are serving in the brain (as opposed to a simpler 3-factor learning mechanism, for instance)?\n\nMinor concerns:\n\n1.\tAll figures have text that is too small, and they all have resolution issues.\n\n2.\tIn several places there is some confusion about individual units ‘releasing’ multiple neuromodulators (e.g. Figure 5E). What precisely is meant by this? Typically, neurons release only a single neurotransmitter (this is Dale’s law) though there are some violations. Furthermore, neuromodulator-releasing neurons are typically localized to particular parts of the brain, projecting out broadly to cortex (e.g. the raphe nucleus for serotonin, the nucleus basalis for acetylcholine, the locus coeruleus for norepinephrine, and the substantia nigra and VTA for dopamine). How exactly do the neuromodulatory variables n_k fit into this picture? It is currently unclear on the basis of the text/figures. Relatedly, Lines 182-186 discuss cell-type specialization, but the phrasing is confusing: dopamine neurons are neurons that release dopamine through their own synapses—while it may be that the recurrent units in the present model cause neuromodulator release, it seems much more biophysically accurate to assume that these recurrent units are causing neuromodulator release in other neurons, which project back to the recurrent circuit; hence, it would be inaccurate to call the recurrent units ‘dopamine neurons’ (maybe something more like ‘dopamine center-activating’ neurons would be more appropriate).\n\n3.\tHow much evidence is there that various neuromodulators affect synaptic transmission specifically, as opposed to cellular membrane excitability or other effects? And does it make sense to have all neuromodulators acting in essentially the same way as one another, as is done in this study?\n\n4.\tMost of the elements of figure 1a are not explained, and serve no obvious purpose (e.g. what is the function of the brain schematic or chemical structure in explaining the results of this paper?). Similarly, the schematic in Fig 1b is messy and would benefit from simplification and color consistency. Shouldn’t the neuromodulatory centers reciprocally connect back to the recurrent network? What does the phrase ‘next leading order’ mean in the Fig 1B caption?\n\n5.\tLine 144: is the Appendix section reference correct here? It is not clear to me how this section of the appendix addresses overfitting.\n\n6.\tCan a baseline be provided for Figure 5? Given the inability of the nmRNN to outperform some baselines in Figure 3, one would expect that figure 5 would at least be required to show improved performance relative to a vanilla RNN in order to justify the claim that the e-nmRNN supports multitask learning.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is original, as far as I can tell, and very thorough in its empirical analyses demonstrating the merits of the e-nmRNN architecture across a variety of domains. Because of the complexity and breadth of experiments performed, the paper frequently suffers due to a lack of clarity, with many important ideas and experimental details buried in the supplementary material. It would benefit most from rigorous edits for clarity. I have serious reservations about the idea that distinct neuromodulators may support compositional generalization across a diversity of tasks (discussed below), and the paper would benefit from greater engagement with the experimental literature and an explanation of testable predictions made by the authors’ model to alleviate these concerns.", "was_revised": true, "final_justification": "The authors have addressed most of the important issues that I raised in my reviews. In particular, they have clarified how temporal credit assignment works in their model, have provided a much-needed baseline, and have clarified some of the confusing language used in the original submission. As a consequence, I've increased my confidence in my score.", "rebuttal": "We thank reviewer n2dL for the insightful and thorough review. We are glad you found the work \"original... and very thorough in its empirical analyses.\" We have worked to address your important concerns about clarity, justification, and the connection to experimental neuroscience.\n\n**Major Concern 1: How does temporal credit assignment work?**\n\nThis is a crucial point that we abbreviated in the main text. Our learning rule is a temporally truncated approximation to the full BPTT gradient. For a parameter like $T_{ijk}$, the update is proportional to:\n$$ \\Delta T_{ijk} \\propto n_k(t) \\cdot \\epsilon_i(t) \\phi'(A_i(t)) r_j(t) $$\nThis rule is \"context-localized\" by $n_k(t)$ and \"temporally-localized\" by using the instantaneous error signal $\\epsilon_i(t)$. While this truncates the full gradient through time, we argue it is a powerful and plausible approximation for two reasons. First, similar to algorithms like e-prop, the recurrent state $\\vec{r}(t)$ and, critically, the slow neuromodulatory state $\\vec{n}(t)$ act as eligibility traces, carrying the necessary temporal context from the past. Second, our empirical results, especially on the RL and multitasking benchmarks which require effective credit assignment over intermediate timescales, demonstrate that this approximation is potent enough to solve complex tasks. The network learns to encode the necessary long-term information into its slow context variables, allowing the local update rule to be effective.\n\n**Major Concern 2: Why neuromodulators? Low-dimensionality and testable predictions.**\n\nThis is an excellent and deep question.\n\n* **Why NMs & Low-Dimensionality**: We chose NMs as our inspiration because their biological function—globally and diffusely reconfiguring circuits—is an exemplar physical substrate for the computational concept of \"context.\" We agree that a handful of NMs are insufficient to capture the full complexity of brain-wide state control. We frame our model as a proof-of-principle showing how even a very low-dimensional modulatory system can powerfully gate high-dimensional dynamics. We agree with your suggestion that this system likely works in concert with other mechanisms (like thalamocortical gating) to form a richer context space. Our goal is to show how volume transmission provides a powerful, efficient, and interpretable means of broadcasting context but, like all models of neural dynamics, this is at best only part of the story.\n* **Testable Predictions**: Our model makes several concrete, testable predictions. 1) As shown in Fig 5C, it predicts that during multitask learning, ambient NM concentrations in a relevant cortical area should become decodable for task context, a hypothesis that is becoming directly testable with new imaging techniques (e.g., GRAB sensors). 2) The model predicts a relationship between the learned timescales of different NM systems and the statistical regularities of the environment. 3) In RL settings, it predicts that some component of the neuromodulatory state space should explicitly encode RPEs, consistent with known roles for dopamine and norepinephrine.\n\n**Minor Concerns:**\n\n* **NM Release/Dale's Law**: We agree our phrasing should be improved. In a revision, we would state that our model is a functional abstraction which can be interpreted in neurobiological terms. A unit \"releasing\" an NM means its activity drives the release of that NM systemically. We will rephrase \"dopamine neuron\" to \"a unit whose activity promotes dopamine release as indicated by the non-zero entry on the $R$ matrix\". Indeed, the one-to-one mapping of nm0 to \"dopamine\" is speculative as an evocative example, but we acknowledge that it may do more harm than good in this context. If given the opportunity to revise, we would remove it in favor of more precise language.\n\n* **Baseline for Figure 5**: This is a critical point we have now addressed. We trained a parameter-matched standard RNN on the multitasking benchmark which have matched previous reported results in Yang 2019. The results show that our spatially-embedded e-nmRNN achieves comparable performance across the 20 tasks when similar effort is put into hyperparameter tuning (using a Bayesian Search with a Tree Structured Parzen estimator). This demonstrates that the inductive biases of our architecture provide tangible value on this complex benchmark, by yielding emergent, interpretable structures (modularity, timescale separation) that a standard RNN does not as readily admit. Indeed, we hypothesize that there is value to more closely matching the inductive biases inherent to neurobiological implementation for both understanding and application.\n\nWe believe these clarifications and new results substantially improve the paper and directly address your thoughtful concerns. Thank you once again for your thoughtful review of our work and your contributions to its improvement."}]

Author rebuttal:

Meta-review: Most reviewers agreed that the proposed method and reported results were innovative and interesting. 

Reviewer 8PJy pointed out that certain assumptions may lack biological plausibility.

In addition, there were some issues about clarity and lack of definitions/decriptions.

My personal inclination is towards acceptance, due to the novelty of the approach and the interesting results (the emergent meta-learning abilities, reward prediction error signals, etc.), provided that the clarifications demanded by the reviewers are included.
Decision: Accept (poster)