Title: Shaping Sequence Attractor Schema in Recurrent Neural Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: schema

attractor

behavioral shaping

sequence
Abstract: Sequence schemas are abstract, reusable knowledge structures that facilitate rapid adaptation and generalization in novel sequential tasks. In both animals and humans, shaping is an efficient way for acquiring such schemas, particularly in complex sequential tasks. As a form of curriculum learning, shaping works by progressively advancing from simple subtasks to integrated full sequences, and ultimately enabling generalization across different task variations. Despite the importance of schemas in cognition and shaping in schema acquisition, the underlying neural dynamics at play remain poorly understood. To explore this, we train recurrent neural networks on an odor-sequence task using a shaping protocol inspired by well-established paradigms in experimental neuroscience. Our model provides the first systematic reproduction of key features of schema learning observed in the orbitofrontal cortex, including rapid adaptation to novel tasks, structured neural representation geometry, and progressive dimensionality compression during learning. Crucially, analysis of the trained RNN reveals that the learned schema is implemented through sequence attractors. These attractor dynamics emerge gradually through the shaping process: starting with isolated discrete attractors in simple tasks, evolving into linked sequences, and eventually abstracting into generalizable attractors that capture shared task structure. Moreover, applying our method to a keyword spotting task shows that shaping facilitates the rapid development of sequence attractor-like schemas, leading to enhanced learning efficiency. In summary, our work elucidates a novel attractor-based mechanism underlying schema representation and its evolution via shaping, with the potential to provide new insights into the acquisition of abstract knowledge across biological and artificial intelligence.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a method for training recurrent neural networks (RNNs) on an odor-reward association task using a ‘shaping procedure’. This procedure, inspired by how rats are trained on the same task, is implemented in three stages. The authors show that shaping leads to faster learning compared to training without shaping, promotes the development of structured task representations, and results in a progressive reduction in the dimensionality of network activity over training. They further analyze the network dynamics, showing a transition from isolated discrete attractors to sequential attractors, and eventually to more abstract sequence attractors. This progression is proposed as a novel perspective on schema formation in biological neural systems. The approach is also applied to a keyword spotting task, where similar patterns emerge, supporting the generality of the findings.\n\nSTRENGTHS AND WEAKNESSES:\nThe structuring of shaping as a learning path is interesting and well thought out. It's particularly nice that the approach is grounded in experimental data and aims to closely follow and be compared to the animal training protocol. However, in its current state it provides only limited evidence that shaping is crucial to achieve the observed properties such as abstract structured representations but mainly shows that also with shaping these properties emerge.\n\nThe concept of shaping described in the paper is closely related to curriculum learning in machine learning, where tasks are broken down and gradually made more difficult over training to facilitate and guide learning. This connection is currently not very clearly referenced or discussed. \n\nThe paper is well structured and some minor clarifications would help to avoid some misunderstandings:\n* Lines 84 and following: It would be helpful to state more clearly that the initially learned values associated with different odors do not persist throughout the sequences. This point only becomes fully clear later in the text or by closely analyzing the figure, which cause a lot of confusion for me.\n* Please clarify what specific behavior was expected from the rat at each training stage. A more explicit description would make it easier to compare the experimental protocol with your model and better understand how the shaping stages align.\n* Line 124: You mention that Gaussian noise was added, but it would improve clarity to include this term directly in Equation 1 (and/or Equation 2), so the reader can see clearly how noise is incorporated into the model dynamics.\n\nQUESTIONS:\n(1) Figure 2(e) shows that shaping improves the correlation between the representational dissimilarity matrices (RDMs) of the model and the data. However, even without shaping, the model still achieves a relatively high correlation and also, a high correlation does not immediately mean that it is a well structured representation. Could you explore or provide another way to demonstrate that shaping is essential for developing a well-structured representation? For example, is there a qualitative difference in the types of attractors or representations formed with vs. without shaping?\n\n(2) The paper shows that shaping leads to a reduction in the dimensionality of the network's activity. Could you also show whether a similar reduction does not occurs in models trained without shaping? This would clarify whether the reduction in dimensionality is specifically a result of shaping or a more general consequence of learning.\n\n(3) In the first stage of training, the RNN is trained to output the odor identity in addition to reward/non-reward. This seems to differ from the animal experiment or were rats also required to classify all 16 odors, or was the task limited to a binary choice (rewarded vs.  non-rewarded)? If the animals were not trained to output odor identity, please explain more clearly why this added output was included in the model and why the comparison to experimental behavior remains valid. Also, would the shaping procedure still be effective if the model were only required to predict reward/non-reward, as in the behavioral task?\n\n(4) You state that shaping leads to more efficient or faster learning (learning-to-learn, Fig., 2(b)). Did you control for how many entirely new odors are introduced in each subsequent problem? If the number of novel odors decreases over time, this could itself contribute to faster learning as the task effectively get easier (less new odors) rather then the network better at learning. Please consider adding a control to distinguish these effects.\n\n(5) L215ff: In this section, you analyze attractor convergence by turning off all inputs. However, since the attractors depend on input, this approach may not reflect where the attractors are under task-relevant conditions. Please clarify this analysis and how it relates to behaviorally meaningful network states. Also, does this approach imply that you are training or tuning the initial conditions of the network during training? If so, how does this relate to the experimental setup with rats?\n\nLIMITATIONS:\n(see questions above)", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates how schema representations can emerge in recurrent neural networks (RNNs) through a shaping-based training protocol. Inspired by experimental paradigms from neuroscience, the authors use a three-stage curriculum (task primitive learning, task sequence learning, and task schema learning) to train RNNs on an odor-based sequential decision task originally developed for rodents. Through this process, the RNN develops low-dimensional sequence attractors that evolve over the shaping phases: from discrete odor-reward associations, to temporally linked sequence attractors, and ultimately to abstract attractor manifolds that generalize across different odor sets. These dynamics replicate key features of schema learning observed in the orbitofrontal cortex, including learning-to-learn behavior, structured representational geometry, and dimensionality compression. The authors further demonstrate that the same attractor-based shaping method improves sample efficiency in a real-world keyword spotting task, suggesting broader applicability beyond the original biological domain.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: This presents a well-executed series of experiments that investigate how shaping—implemented as a staged curriculum—sculpts internal dynamics in RNNs trained on a biologically inspired sequential decision task. The authors show that shaping leads to attractor structures at multiple scales: discrete odor-specific fixed points, low-dimensional sequence manifolds, and eventually generalized “schema” dynamics. These results are supported by fixed-point analyses, PCA visualizations, and comparisons to orbitofrontal cortex recordings. The use of a neuroscience-motivated shaping protocol (mirroring rat studies) and the emphasis on mechanistic explanation through dynamical systems tools are strengths.\n\nHowever, while the findings are compelling, the paper does not address whether the learned attractor solutions are unique or degenerate across initializations—a key concern given prior work (e.g., Kepple et al., 2022; Huang et al., 2024) showing that multiple RNNs can exhibit equivalent task behavior but diverge in internal structure. Although the authors mention seed averaging, they do not quantify solution diversity or identifiability, leaving open whether the schema attractor is a convergent or incidental outcome of the shaping process. Acknowledging these issues would improve the paper’s rigor.\n\nClarity: The paper is clearly written and well-structured. The three-phase shaping procedure is motivated, defined, and visually summarized. Core concepts—such as “sequence attractors” and “schema manifolds”—are introduced in accessible terms, and the experimental design is intuitive. The dynamical analyses (e.g., attractor structure, dimensionality compression, RNN trajectory geometry) are presented coherently, even for readers outside of neuroscience. Still, the framing of novelty could be made more explicit. Key terms such as “sequence attractor schema” are introduced without a formal definition, and the distinction from earlier work on compositional dynamics, schema learning, or task manifolds (e.g., Yang & Rajan 2019; Kepple et al. 2022) is somewhat implicit. Greater effort to clarify what is conceptually new—especially relative to existing schema modeling frameworks—would help the paper stand on its own.\n\nSignificance: The submitted work addresses a long-standing question in both neuroscience and machine learning: how structured representations and learning-to-learn capabilities can emerge in sequential environments. The use of curriculum (shaping) to elicit reusable internal dynamics, and the demonstration that these dynamics resemble neural phenomena in OFC, make the paper significant for biologically grounded modeling. The generalization of the approach to a keyword spotting task also suggests relevance to applied RNN training. That said, some aspects of the contribution are only confirmatory. The idea that curriculum learning induces reusability and compositionality is well established (in both biological and artificial systems), and prior work has shown that networks can learn task-aligned manifolds and latent dynamics under suitable conditions. Without quantifying generalization across domains or linking the observed schema attractors to improved task-level abstraction or zero-shot performance, the submission’s broader implications remain suggestive but incomplete.\n\nOriginality: This submission introduces a new synthesis of ideas. It applies a structured shaping protocol to sculpt attractor-based schema representations in RNNs and connects this process to known neurophysiological dynamics. The concept of progressively building attractors—from primitives to sequences to abstract schemas—via curriculum is well-illustrated and not previously demonstrated in this form. However, several foundational ideas in the paper have appeared in earlier work, including compositionality (Yang & Rajan, 2019), degeneracy in internal solutions (Huang et al., 2024), and shaping effects on internal structure and generalization (Kepple et al., 2022). While the current work builds productively on these foundations, the submission does not fully acknowledge this overlap. A clearer positioning would help delineate what is newly shown (e.g., sequence-level attractors shaped by curriculum) versus what conceptually recurs under different names.\n\nQUESTIONS:\nThe paper shows that shaping yields structured schema dynamics, but it is unclear whether these attractors emerge consistently across random initializations. Have the authors analyzed whether different networks trained under the same shaping protocol converge to similar internal solutions (e.g., attractor geometry, transition structure), or whether they are one of many functionally equivalent but structurally distinct implementations? Quantifying this could clarify whether the observed schema dynamics are necessary or incidental. Prior work (e.g., Huang et al., 2024; Kepple et al., 2022) highlights this concern.\n\n\nThe framing of \"sequence attractor schema\" is compelling, but conceptually overlaps with earlier accounts of schema learning and compositional representations in RNNs (e.g., Yang & Rajan, 2019; Driscoll et al., 2024). Could the authors clarify how their formulation differs from those prior works — for instance, in terms of mechanisms, representations, or generality — and what is uniquely demonstrated here?\n\n\nThe shaping procedure appears to follow a specific three-phase design. How sensitive are the results to the structure or granularity of this curriculum? For example, would adding more intermediate steps or using a less biologically motivated sequence of subtasks still produce schema attractors? Clarifying this could help assess whether shaping serves primarily as a biological model or as a general training principle.\n\n\nWhat is the role of attractor geometry in generalization? The shaped RNN exhibits low-dimensional attractor dynamics and faster learning on novel sequence tasks. Do the authors have evidence that the geometry of these attractors (e.g., manifolds, fixed points, transitions) directly contributes to generalization, beyond faster convergence? For instance, are new sequences dynamically composed from existing substructures? A deeper analysis here could clarify the functional role of the schema attractor beyond representational structure.\n\n\nWhat is the relevance to broader RNN architectures and tasks? The results are demonstrated in a vanilla RNN trained on a neuroscience-inspired sequence task and briefly extended to keyword spotting. Could the authors comment on whether they expect shaping to induce similar schema dynamics in larger or more structured RNNs (e.g., GRUs, LSTMs, gated architectures, or multi-region RNNs as in the recent neuroscience literature), or in non-sequential domains? Understanding these boundaries would clarify the scope of the paper’s claims and implications.\n\nLIMITATIONS:\nYes, though several important contextual gaps remain. The submission presents strong evidence that shaping induces structured, schema-like dynamics in RNNs, but does not address whether these dynamics are consistent across random seeds or training runs. Prior work (e.g., Huang et al., 2024; Kepple et al., 2022) has shown that multiple recurrent network solutions can yield the same behavior while differing internally, raising questions about identifiability and robustness. A discussion of this variability, even in the absence of new experiments, would help calibrate how general the observed attractor structure is.\nIn addition, while the concept of curriculum shaping is well-motivated, its relation to earlier theoretical and modeling work on schema learning, compositional dynamics, and attractor reuse is not fully articulated. Greater clarity on what is conceptually new would strengthen the paper’s broader positioning. Finally, the extension to domains beyond the primary odor-sequence task is limited, leaving open whether the shaping protocol generalizes across architectures or learning settings. These are not fatal flaws, but should be acknowledged in any discussion of the work’s scope.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInspired by rat study on odor-sequence task (Zhou et al), the authors model schema learning using RNNs trained with shaping. They describe a 3-stage learning process: first, learning basic task primitives (classification of odor identity and reward); then combining them into sequences (reward only); and finally training on a generalized version of the task (reward only). The trained RNNs show hidden state representations similar to the neural activity in rat OFC. This is shown by RDM analysis, and progressive dimensionality compression (PCA). They also reverse-engineer the RNN, showing it first develops discrete attractors associated with each odor, which are later reused to form sequences. Finally, they generalized the 3-stage shaping learning process to additional experiments of keyword spotting.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n The paper addresses a very interesting topic, shaping in training, which is indeed critical to both animal and artificial learning. In practice, certain curricula/ shaping are often essential to achieve high behavioral accuracy. \nThe resemblance between the RNN and empirical data seems convincing.\nThe effect of curriculum on emergence of dynamical objects has been studied, but not extensively – so this is a very welcome contribution.\n\n\nWeakness\n1.\tThe paper focuses more on replicating empirical observations of Zhou et al than on exploring why shaping is necessary. While similarities between RNNs and neural representations in animals are well documented in past, the mechanisms that make shaping essential remain unclear—an open question that could significantly advance the field.\n2.\tI'm not fully convinced that only the specific 3-stage training procedure can produce the observed similarity between RNNs and rats. The reuse of attractors has been described in previous work (Driscoll et al Nature neuro 2024, Turner et al Neurips 2023 ). This raises the question of whether similar results would emerge under other training protocols not explicitly following the 3-stage design (Control conditions are missing).\n3.\tThe final experiments, intended to demonstrate generalization of the 3-stage shaping approach, seem too similar to the initial task. It’s unclear what truly distinguishes them or how much generalization is actually being tested.\n4.\tSome training details are different from those of the experiment, and many details seem arbitrary. This raises the question of how robust are the results to these specific choices. Questions below detail these issues.\n\nQUESTIONS:\n1.\tIn the Zhou et al paper, the first paragraph highlights the special role of odors 13-14. It’s very easy to miss this detail in the current paper. I think it’s important to emphasize it when describing the task.\n2.\tYou state that “Zhou et al. found it very challenging to train rats to learn the task directly, so they adopted a shaping approach”. In the Zhou et al paper, they describe shaping in Methods, but don’t mention anything about challenges. \n3.\tUnlike rats, RNNs are easily manipulable, so it’s unclear why the authors didn’t leverage this to better understand why shaping is so effective or design improved shaping strategies that could be tested in animal experiments. For example, could training in an interleaved fashion—alternating between L1 and L2—speed up learning?\n4.\tFrom my understanding, all weights are updated throughout the 3 training stages. How would the results change if certain layers were frozen and not modified across stages?\n5.\tWhat is the overall performance of the control model? How does the loss curve of this model compare to the shaping model? The RDM matrix is less similar, but it would still be useful to see it (and the dendrogram of Figure S2).\n6.\tCould the reduced similarity between the control model and the empirical rat data be due to the P4–P5 odors (13–14) which are reversed and not trained for identification as in the shaping RNN model? Also, it’s unclear why this task choice was made—some details seem to be missing (I’m aware the original study did the same reversing).\n7.\tWhy use 20 for the non-zero entry and 2 for the desired output?\n8.\tWhy is learning rate in schema an order of magnitude slower than task primitive?\n9.\tThe Zhou experiment always moves from S1 to S2 to S1 and so on. Why not do the same in simulations?\n10.\tThe difficult odor pair 13-14 is treated differently in the shaping in the Zhou experiment. Training is first done on S1 odors, then on 13-14 (including reversals), and then on the rest of S2. Why not do the same in simulations?\n11.\tWhy include odor output in the first stage and not in the second one? Behavior doesn’t require odor identity in either stage. If the OFC always outputs identity, it should continue doing so in all stages.\n12.\tFigure 2B. The legend “experiment fit” suggests that this is data from the Zhou experiment. If I’m not mistaken, this is just a fit to the simulations. Also – does this fit contribute to understanding the results? Also – the caption says polynomial, and the main text exponential.\n13.\tLine 168: “This pattern mirrors learning-to-learn behavior observed in rats performing the odor-sequence task.”. It is not that easy to find this in the Zhou paper. In fact, extended figure 1 in the Zhou paper does not appear to show any evidence of faster learning. Figure 5 in that paper is the closest I could find, which shows changes the rate of changes in latency – but not as a function of problems.\n14.\tSupp Line 33 “Each major branch is then further divided into six sub-branches according to spatial position.” First, there are less than 6 options in each major branch. Second, I don’t see this in Figure S2. \n15.\tFigure S4: Why is smaller learning rate for “Ours” a fair comparison? The measure is number of trials.\n16.\tFigure S6 panel C. Typo. Cross – corss \n17.\tThe results of Figure S7 are intriguing, suggest new experiments, and perhaps should be in the main text.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a computational model of schema formation through shaping, inspired by neurocognitive processes. Using a curriculum learning paradigm modeled after animal studies (specifically odor-sequence tasks in rats (REF 4)), the authors train RNNs to form discrete attractors, link them into sequential dynamics, and ultimately abstract these into low-dimensional sequence attractors representing schema. The model replicates key experimental findings from orbitofrontal cortex (OFC) studies, including faster learning (learning-to-learn), dimensionality compression, and structured representation geometry. The framework is also tested on a keyword spotting task to demonstrate broader applicability. The manuscript is clearly written.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper introduces a novel dynamical systems account of schema formation, grounded in neuroscience and with relevance for both computational neuroscience and machine learning. The shaping paradigm is directly motivated by animal training literature and aligned with known properties of the prefrontal cortex and OFC. The RNN replicates key neural signatures observed in OFC recordings, including the progressive compression of dimensionality and structured representational dissimilarity matrices (RDMs), with high correlation to empirical data. The paper provides a compelling narrative of schema evolution: from discrete point attractors → sequence attractors → abstract attractors.  \n\nWeaknesses: \n-It's unclear how architecture-dependent the attractor formation is. For example, do gated RNNs (LSTM/GRU) or Transformers exhibit similar attractor dynamics under shaping?\n\n-Most results rely on a single neuroscience-inspired task. The keyword spotting task is a welcome addition, but further validation on more varied or large-scale datasets (e.g., navigation or language tasks) would demonstrate scalability and robustness.\n\nQUESTIONS:\n1- From Fig. 4c, it looks like LSTM has much better performance in terms speed of learning. Can authors elaborate the reason for this? And if this is the case, why did the authors choose to work with RNNs rather than LSTMs?\n\n2- In a similar vein, do LSTMs have similar attractor dynamics when solving this task?\n\n3- I understand the timing constraint in running extra models, but can authors compare their current model and training strategy with other recent models such as meta-reinforcement learning model (Yang, Nature Neuro, 2018)?\n\n4- the analysis in Fig. 4g-i is very cool. I am wondering what is the difference of this trajectory between \"with shaping\" and \"without shaping\" networks. In other words, do both types of networks follow similar trajectories to get to abstract state or these is fundamental difference between them?\n\nLIMITATIONS:\nI agree with authors that limitations of this work are reliance on back-propagation and lack of multi area interactions. Those are good avenues for future improvements.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}