Title: Volume Transmission Implements Context Factorization to Target Online Credit Assignment and Enable Compositional Generalization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Meta-learning

Bio-inspired

Hypernetworks

Compositionality

Sequence Learning
Abstract: The modern connectivist framing of neural computation emphasizes the primacy of synaptic communication at the risk of neglecting the influence of the surrounding neuromodulatory environment --- a neuron's 'biophysical context.' Decades of experimental work has established two views of neuromodulatory (NMs) influence: 1) NMs significantly alter circuit dynamics and 2) NMs gate synaptic plasticity, acting as a 'third factor' in learning. Here, we unify these perspectives, proposing that neuromodulation via volume transmission implements a powerful computational principle: context factorization. We derive an endogenously neuromodulated Recurrent Neural Network (e-nmRNN) from a rate reduction of NM release, showing how NM concentrations dynamically factorize network connectivity. This framework reveals how multiplicative NM gating distinctly influences dynamical regimes compared to additive input. Crucially, this context factorization enables targeted online credit assignment: learning rules derived for the e-nmRNN are naturally gated by NM concentrations, localizing updates to relevant contexts. We demonstrate that e-nmRNN dynamics can learn to approximate gradient descent, facilitating rapid in-context adaptation akin to meta-learning. Empirically, e-nmRNNs achieve strong compositional generalization in sequence-to-sequence tasks, outperforming baselines and exhibiting greater hyperparameter robustness. Furthermore, when trained on complex multitasking benchmarks, e-nmRNNs develop emergent properties mirroring biological observations, including modularity, cell-type specialization based on NM release, and distinct neuromodulatory timescales encoding task context. The model's interpretability allows us to reverse engineer these emergent structures. Notably, in reinforcement learning tasks, the e-nmRNN learns to encode context and signals like Reward Prediction Error (RPE) within its neuromodulator dynamics, demonstrating a mechanism for RPE-gated online credit assignment essential for learning how to learn. By bridging biophysical mechanisms with computational principles and empirical validation, our work presents e-nmRNNs as a performant, interpretable model for understanding the computational role of neuromodulation in flexible and compositional learning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this study, the authors present an ‘endogenously neurmodulated recurrent neural network’ (e-nmRNN), which uses neuromodulatory volume transmission to effectively induce hypernetwork-like dynamics in recurrent neural networks. The gating induced by the authors’ postulated neuromodulators enables in-context adaptation and compositional generalization on a variety of tasks, because gating suppresses unused dynamical motifs and prevents learning (ameliorating issues caused by catastrophic forgetting).\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is original, as far as I can tell, and very thorough in its empirical analyses demonstrating the merits of the e-nmRNN architecture across a variety of domains. Because of the complexity and breadth of experiments performed, the paper frequently suffers due to a lack of clarity, with many important ideas and experimental details buried in the supplementary material. It would benefit most from rigorous edits for clarity. I have serious reservations about the idea that distinct neuromodulators may support compositional generalization across a diversity of tasks (discussed below), and the paper would benefit from greater engagement with the experimental literature and an explanation of testable predictions made by the authors’ model to alleviate these concerns.\n\nQUESTIONS:\nMajor concerns:\n\n1.\tHow does temporal credit assignment work in the learning algorithm employed here? The approach taken seems to be temporally greedy (appendix section 3) and is justified in terms of truncation of the gradient signal to ‘leading order’. However, temporally greedy approaches are in general not a well-justified approximation, and full temporal credit assignment using the backpropagation through time algorithm is required for good performance under most conditions without additional tricks. The e-prop and RFLO algorithms are cited, but these methods are not fully temporally local (they discard cross-neuron temporal interactions but not within-neuron temporal interactions), and the relationship between these methods and the present learning algorithm are unclear. As such, appendix section 3 would benefit from a rewrite emphasizing clarity and the present algorithm’s relationship to previous methods.\n\n2.\tTaking a step back, why were neuromodulators selected as the source of synaptic gain modulation underlying this method? For instance, very similar effects have been achieved via a model of thalamocortical gating (Logiaco et al. 2021), wherein thalamic input modifies the gain of some neurons (or could potentially excite or inhibit select portions of a dendritic arbor). The trouble with neuromodulators is that there are only a handful of distinct types: in this study it seems as though four were selected for most experiments. In contrast, for many types of learning, one could imagine that the relevant compositional primitives could range from tens to hundreds. How could variations in neuromodulatory tone in such a low-dimensional space function to adequately support learning of such a high diversity of primitives? And relatedly, is there any way of experimentally testing that this really is the function that neuromodulators are serving in the brain (as opposed to a simpler 3-factor learning mechanism, for instance)?\n\nMinor concerns:\n\n1.\tAll figures have text that is too small, and they all have resolution issues.\n\n2.\tIn several places there is some confusion about individual units ‘releasing’ multiple neuromodulators (e.g. Figure 5E). What precisely is meant by this? Typically, neurons release only a single neurotransmitter (this is Dale’s law) though there are some violations. Furthermore, neuromodulator-releasing neurons are typically localized to particular parts of the brain, projecting out broadly to cortex (e.g. the raphe nucleus for serotonin, the nucleus basalis for acetylcholine, the locus coeruleus for norepinephrine, and the substantia nigra and VTA for dopamine). How exactly do the neuromodulatory variables n_k fit into this picture? It is currently unclear on the basis of the text/figures. Relatedly, Lines 182-186 discuss cell-type specialization, but the phrasing is confusing: dopamine neurons are neurons that release dopamine through their own synapses—while it may be that the recurrent units in the present model cause neuromodulator release, it seems much more biophysically accurate to assume that these recurrent units are causing neuromodulator release in other neurons, which project back to the recurrent circuit; hence, it would be inaccurate to call the recurrent units ‘dopamine neurons’ (maybe something more like ‘dopamine center-activating’ neurons would be more appropriate).\n\n3.\tHow much evidence is there that various neuromodulators affect synaptic transmission specifically, as opposed to cellular membrane excitability or other effects? And does it make sense to have all neuromodulators acting in essentially the same way as one another, as is done in this study?\n\n4.\tMost of the elements of figure 1a are not explained, and serve no obvious purpose (e.g. what is the function of the brain schematic or chemical structure in explaining the results of this paper?). Similarly, the schematic in Fig 1b is messy and would benefit from simplification and color consistency. Shouldn’t the neuromodulatory centers reciprocally connect back to the recurrent network? What does the phrase ‘next leading order’ mean in the Fig 1B caption?\n\n5.\tLine 144: is the Appendix section reference correct here? It is not clear to me how this section of the appendix addresses overfitting.\n\n6.\tCan a baseline be provided for Figure 5? Given the inability of the nmRNN to outperform some baselines in Figure 3, one would expect that figure 5 would at least be required to show improved performance relative to a vanilla RNN in order to justify the claim that the e-nmRNN supports multitask learning.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors consider a model of a recurrent neural network enhanced with a simplistic version of volume transmission mediated by neuromodulators. They show that these hybrid networks can be trained to solve certain representative tasks that involve aspects of online credit assignment, compositional generalization and a reinforcement learning task.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper considers an often overlooked aspect of neural network function, and is a step towards a deeper understanding of how volume transmission when combined with classical synaptic communication leads to novel and more efficient forms of neural computation. \n\nWeaknesses: The paper is incomplete in its current form, and significant revisions (well beyond the scope of this submission) are necessary to bring it to par. \n\nFirst, the main text contains equations where a significant number of the variables, parameters and abbreviations are not defined. One should not have to read the appendix to know what a variable represents. To give a few examples (which is certainly not an exhaustive list), \n1. LIF is not defined in line 73\n2. In the equation at the end of page 2, S is not defined. \n3. In the equations above line 74, the nonlinearities \\phi and \\theta are not defined. In the Appendix, different symbols are used to represent these quantities, and that makes it further confusing. \n4. In the equation above 87, D is not defined (and later D is also used for dimensionality)\n5. In the equation above line 121, A and \\epsilon are not defined. \n\nSecond, the Appendix has trailing sentences that are left incomplete -- for example, lines 43 and 44. It is also poorly organized -- the derivation of the final model from the LIF model is hard to follow (e.g. unnecessary one-sentence subsections, the function phi appears several times before being derived, etc.)\n\nThird, the main text has several grandiose statements about the framework addressing important biological questions rather than critical details on the tasks and training of the models. Currently, the main text is unspecific to the point where it is unclear what is being done with this model. A few examples:\nFigure 5: Unclear what the 20 tasks are and how the model is trained on all of them. \nFigure 3: The distinction between pre-training (trained on a fixed training set of sine curve parameters?) and in-context learning (new combinations of sine curve parameters not in the training set?) is unclear.\nIn Figure 4, the switching of the bandit arm reward probabilities seems almost periodic. It is unclear from the main text/Appendix what parameters are being used. \n\nFourth, the model proposed by the authors involves unrealistic biophysical assumptions. For example, a key assumption in the model is that neuromodulators are well-mixed (i.e., that the mixing time is negligible compared to the timescale of neural computation), but it is unclear how this is physically plausible. Neural activity operates on timescales of a few tens of milliseconds, while diffusion of neuromodulators occurs at much slower timescales. The authors also assume a quadratic \"chemical\" interaction between neuromodulators (last term in eq. 18) which is not justified. What is the biophysical basis behind this interaction term? The authors also add in nonlinearities to the equations (lines 115 and 127 of the Appendix) without a clear justification, and claim that this is a \"modeling assumption\". This is OK if the paper also didn't claim to be biologically realistic (lines 195 to 197 of the Appendix).\n\nQUESTIONS:\nDefine “context factorization”: this is the “central claim” of the paper but this concept is not well explained, so we do not understand what difficult or biologically relevant feature this model is achieving (that previous models did not). \n\nSection 2.2 “Multiplicative gating of dynamics” is unclear. It should either be expanded to properly explain the two distinct bifurcation mechanisms, or cut altogether to focus on explaining better the results in subsequent sections.  The discussion has a more precise statement about this than the results section.  \n\nFigure 3: the results are not very impressive, Oscillatory RNNs and gated recurrent units overperform. Comment on this aspect, and please explain why some models overperform. \no\tIs this the “overfitting” issue? Also please explain this issue better, identify which parts of which figures show overfitting, and propose concrete solutions – model reduction, bigger training sets?  \n\t\nFigure 4: test a bandit where the correct arm switches more unpredictably; here, especially with the anticipatory switching, it just looks like the RNN happens to oscillate at the “right” frequency to match the reward switching frequency, without tracking it. \no\tMaybe repeat for increasingly stochastic switching (vary both the switching frequency and the inter-switch duration variability) and plot performance versus these stochastic switching parameters?\n\nFigure 5: it would be interesting to plot the spatial localization of neuromodulator release. Make a plot of cells at their 2D positions and color them according to their average or instantaneous release of a neuromodulator. \no\t Relatedly: do cells of the same type colocalize to different neighborhoods (thus concentrating their range-limited – eq. in panel A –  synaptic interactions within their subgroup) or are they spread out to interact homogeneously with other cell types? \n\nFigure 5: poor resolution, heatmaps of panel E have meaningless ticks.\n\nLIMITATIONS:\nThe limitations section is currently brief and non-specific -- this could benefit from more precise statements, perhaps commenting on specific modeling assumptions or the set of tasks considered in this work. For instance, the well-mixed approximation, the various steady-state approximations from the LIF model to the trainable e-nmRNN, where all interactions are tensor products (non-linear dynamics though arise from multiplying n(t)r(t)) inside a global nonlinearity could be discussed. Also, please propose more concrete solutions for the overfitting problem stated here.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a biologically inspired recurrent neural network, e-nmRNN, motivated by the central role of neuromodulation and volume transmission in the brain. The model operationalizes the concept of context factorization, wherein neuromodulator concentrations dynamically gate network connectivity, providing an interpretable mechanism that links biological mechanisms and computational principles. Through both analytical derivation and empirical evaluation, the paper demonstrates that context factorization enables targeted online credit assignment (with learning updates gated by the neuromodulatory context) and supports compositional generalization. The work includes experiments in supervised learning, meta-learning, and reinforcement learning, showing superior or competitive performance to standard baselines, as well as emergent modularity and cell-type specialization mirroring biological observations.\n\nSTRENGTHS AND WEAKNESSES:\n***Strengths***:\n\n1. The paper is clearly written and easy to follow.\n\n2. The proposed endogenously neuromodulated RNN (e-nmRNN) presents a novel architectural idea. The work systematically relates this model to diverse learning settings and biological phenomena, offering valuable insights into the functional role of neuromodulators in the brain.\n\n3. Experimental results are well-supported by appropriate baselines, and related literature is thoroughly discussed, situating the work well within existing research.\n\n\n\n***Weaknesses***:\n\nTransformers and architectures like Hippo-LegT derive much of their power from strong scalability and parallelization capabilities, which enable them to handle long sequences and large datasets efficiently. In contrast, the scalability properties of e-nmRNN are less clear. It would be valuable for the paper to discuss the computational cost and parallelizability of e-nmRNN, especially in comparison to modern scalable architectures.\n\nQUESTIONS:\n1. The spatially embedded variant outperforms the standard e-nmRNN. Could this improvement be attributed to effective sparsity induced by spatial constraints? \n\n2. In the multi-task decision-making experiments (section 6), does contextual differentiation emerge autonomously from the model dynamics, or is context explicitly provided as input? \n\n3. What precisely differentiates the modeled “dopamine” and “serotonin” neurons in the architecture? Is it solely based on opposite modulation slopes (e.g., positive vs. negative sensitivity) ?\n\nLIMITATIONS:\nsee weaknesses and questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a context factorization algorithm which is inspired by the roles of neuromodulators in the brain to alter circuit dynamics and act as a third factor in Hebbian learning. They incorporate neuromodulators, which have their own dynamics, as an additional factor modulating recurrent connectivity in an RNN, as well as several different classes of decoders. They show that the RNN uses the neuromodulators to approximate gradient descent in-weights, and can generalize compositionally. They then demonstrate its efficacy on RL tasks and that it leads to brain-like solutions in multi-task learning when using a spatial embedding.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n*Quality:*\n- The paper contains a large number of carefully-conducted analyses and simulation settings from supervised learning to RL\n- The algorithm learns to solve meta-learning tasks significantly faster than the baselines\n- The paper demonstrates a strong mix of theory and simulation, illustrated for instance by the analysis of the emergent gradient-descent-like updates in the RNN\n- The paper cites widely and is well-grounded in neuroscience\n\n*Clarity:*\n- The diagrams in the figure help describe the concepts well; I appreciated the text labels to guide the reader\n\n*Significance:*\n- The paper represents a potentially significant advance for machine learning by providing theoretical analyses and simulations demonstrating the utility of multiplicative gating in RNNs, and for neuroscience by using this as a model for neuromodulators\n- It recovers known features of neuroscience such as emerging RPEs\n- The authors detail plans for follow-up projects in neuroscience\n\n*Originality:*\n- As far as I know, the machine learning demonstration of the utility of neuromodulator-like factors for continual learning is novel\n\n**Weaknesses:**\n\n*Quality:*\n\n*Clarity:*\n- Not all variables are defined when equations are presented (although they are later in the list, lines 77-86). Perhaps it is possible to guide the reader better here?\n- The paper requires a significant amount of prior knowledge in neuroscience in order to be easily understood (I would not expect readers to know Dale's law without definition, for instance)\n- nmRNNs are frequently referenced in the paper; it would be helpful to briefly explain how they differ from the e-nmRNNs here\n\n*Significance:*\n-  It would be good to emphasize the paper's original contributions a bit more, perhaps in a bulleted \"contributions\" list in the introduction, to make clear what its additions are to prior work\n\n*Originality:*\n- As the authors acknowledge, representing neuromodulators as gating is not a new idea in NeuroAI\n\nQUESTIONS:\n1. How does an e-nmRNN differ from an nmRNN (in words and equations)? What is the particular advantage of the \"endogenous\" part?\n\n2. Are there situations in which you would expect the algorithm to fail?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}