Shaping Sequence Attractor Schema in Recurrent Neural Networks
Authors: Zhikun Chu
Bo Hong
Xiaolong Zou
Yuanyuan Mi
Venue: NeurIPS 2025 poster ()
Keywords: schema
attractor
behavioral shaping
sequence
Primary area: neuroscience_and_cognitive_science
TL;DR: This study reveals that shaping progressively builds attractor dynamics in RNNs for generalizable sequence schemas, facilitating efficient learning and abstraction.

Abstract: Sequence schemas are abstract, reusable knowledge structures that facilitate rapid adaptation and generalization in novel sequential tasks. In both animals and humans, shaping is an efficient way for acquiring such schemas, particularly in complex sequential tasks. As a form of curriculum learning, shaping works by progressively advancing from simple subtasks to integrated full sequences, and ultimately enabling generalization across different task variations. Despite the importance of schemas in cognition and shaping in schema acquisition, the underlying neural dynamics at play remain poorly understood. To explore this, we train recurrent neural networks on an odor-sequence task using a shaping protocol inspired by well-established paradigms in experimental neuroscience. Our model provides the first systematic reproduction of key features of schema learning observed in the orbitofrontal cortex, including rapid adaptation to novel tasks, structured neural representation geometry, and progressive dimensionality compression during learning. Crucially, analysis of the trained RNN reveals that the learned schema is implemented through sequence attractors. These attractor dynamics emerge gradually through the shaping process: starting with isolated discrete attractors in simple tasks, evolving into linked sequences, and eventually abstracting into generalizable attractors that capture shared task structure. Moreover, applying our method to a keyword spotting task shows that shaping facilitates the rapid development of sequence attractor-like schemas, leading to enhanced learning efficiency. In summary, our work elucidates a novel attractor-based mechanism underlying schema representation and its evolution via shaping, with the potential to provide new insights into the acquisition of abstract knowledge across biological and artificial intelligence.

Shaping Sequence Attractor Schema in Recurrent Neural Networks
Zhikun Chu' wow.t//n.iioottooo.cooo com
Bo Hong hongbo@tsinghua. edu, cn
Xiaolong Zou benzou@mail. bnu, edu. cn
Yuanyuan Mi
miyuanyuanttsinghua edu.
1, Bioenngieerring College, of Medicine, Chongqing University.
2, Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China.
3, Biomedical Engineering.Schun of Medicine, Tsinghua University.
M: Corresponding authors.
Abstract
Sequence schemas are abstract, reusable knowledge structures that facilitate rapid adaptation and generalization in novel sequential tasks. In both animals and hu- mans, shaping is an efficient way to acquire such schemas, particularly in complex sequential tasks. As form of curriculum learning, shaping works by progressively advancing from simple subtasks integrated full sequences, and ultimately en- abling generalization across different task variations. Despite the importance of schemas cognition and shaping in schema acquisition, the underlying neural dynamics at play remain poorly understood. To explore this, we train recurrent neural networks on an odor-sequence task using shaping protocol inspired by well-establisheds paradigms in experimental neuroscience. Our model provides the first systematic reproduction of key features of schema learning observed the orbitofrontal cortex, including rapid adaptation to novel tasks, structured neu- ral representation geometry, and progressive dimensionality compression during learning. Crucially, analysis of the trained RNN reveals that the learned schema is implemented through sequence attractors. These attractor dynamics emerge gradually through the shaping process: starting with isolated discrete attractors in simple tasks, evolving into linked sequences, and eventually abstracting into generalizable attractors that capture shared task structure. Moreover, applying our method to keyword spotting task shows that shaping facilitates the rapid development of sequence attractor schemas, leading to enhanced learning efficiency. In summary, our work elucidates a novel attractor-bac based mechanism underlying schema representation and its evolution via shaping, offering new insights into the acquisition of abstract knowledge across biological and artificial intelligence.
Introduction
Imagine taking the subway in new station: you can effortlessly anticipate the sequence of events entering the station, purchasing a ticket, scanning it, waiting for the train, and boarding. This ability stems from an abstract knowledge structure encoded in your brain [I] which organizes the typical order and relationship among these events, commonly referred to as a schema. Schemas facilitate
rapid learning ( 3 flexible siccision-mkinng 4. and efficient generalization 6 7 forming the
foundation of cognitive flexibility and generalization in both animal and human intelligence [ How-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Task Primitive Leaming
Task Sequence Learning
Task Schema Learning
b
Discrete Altractors
Sequential attractors
Abstract attractors
d
P2
P3
P8
Probeen
Reward
Non Reward
S1a
Profre22
31b
Proteen
S20
52b
Figure I: Schematic of learning sequence schema via Shaping. (a) A dolphin example. Task primitive learning: the dolphin learns basic actions independently, such as holding ball and passing through hoop. Task sequence learning: it learns to order these actions temporally holding the ball first, then passing through the hoop. Task schema learning: exposure to different ball and hoop types allows the dolphin to develop generalizable schemas, enabling adaptation to similar tasks. (b) A neural dynamical hypothesis. Task primitive learning: Neural systems acquire basic attractor structures (e.g., discrete attractors). Task sequence learning: These discrete attractors are temporally linked into structured sequence attractors. Task schema learning: These scquence-specific attractors are compressed into abstract schema. (c) Task primitive learning: An odor-reward association task presented where 16 odors are linked to different rewards (red = rewarded, blue = non-rewarded). (d) Task sequence learning: Using the 16 odors, an odor-sequence task is constructed with two pairs of six six-position positioi sequences (Sla-S1b and S2a-S2b). Arrows show transitions between and within sequences. (e) Task schema learning: Five new problems with the same sequence structure but novel sets of 16 odors are introduced consecutively.
ever, despite their importance, the neural mechanisms underlying the representation and acquisition of schemas remain poorly understood.
For simple tasks, schemas can learned through direct trial-and-error training on related experiences. However, for more complex sequential tasks, this approach often fails. Instead, animals and humans typically rely on shaping
process that decomposes complex tasks into simpler subtasks learned incrementally 88, 9 10]. As illustrated in Fig. [. teaching dolphin to jump through a hoop while holding a ball exemplifies this approach. The animal first learns each basic skill independently (e.g.. jumping through hoop, holding a ball), then practices performing them in sequence, and ultimately generalizes the behavior across task variations (e.g., different ball colors or hoop types). Schema learning via shaping typically unfolds in three stages [4] (1) task primitive learning. where basic action components are acquired; (2) task sequence learning, where these components are integrated into structured behavioral sequences; and (3) task schema learning, where abstract regularities are extracted across multiple structurally related tasks. While shaping has proven effective practice, how it supports the learning and representation of task schemas remains unclear.
Schemas, primarily represented in the prefronnal cortex (PFC) I M3 are characterized by two key properties: low dimensionality and compositionality. The PFC is thought to support rich, low- dimensional attractor dynamics that underlie flexible behavior for example, continuous attractor dynamics for sensory integration [I]] and discrete attractors for sensorimotor transformations [12 These low-dimensional dynamics have been proposed as neural representations of schemas
[3] and
are thought to underlie the carning-to-lean phenomena observed in primates [ Schemas are also compositional, allowing them to reused and recombined to solve novel tasks
For instance, rats
can transfer learned spatial schemas from one context to novel memory tasks
Moreover, recent

---

computational studies demonstrate that basic attractor dynamics, arising from multitask learning, can serve as primitive schemas that flexibly combine to facilitate new task learning (2 15].
Building on these propoties, we propose dynamical perspective on schema formation through shaping in complex sequential tasks (Fig. [Tb), consisting of three stages: (1) Task primitive learning establishes basic attractors for individual task components, such as discrete attractors; (2) Task sequence learning integrates them into structured sequential dynamics, such as sequential attractors; (3) Task schema learning abstracts and compresses these patterns into unified low-dimensional representations that capture shared temporal and structural regularities.
Although the dynamic view of schema formation is compelling, it lacks a concrete computational model and direct comparison with neural data. In this work, we use recurrent neural networks (RNNs) to investigate how schema representations emerge and evolve through shaping. We validate our approach in an odor-sequence task widely used study schema learning in the orbitofrontal cortex (OFC) Our main results and contributions are as follows.
(I) Our shaping- chaping-trained RNN systematically replicates key features of schema learning observed in rats' OFC ensembles, including faster learning on novel tasks, structured task representation geometry, and progressive dimensionality compression.
(2) The model reveals that schema representations emerge as -ow-dimesioonll sequence attractors, composed of discrete attractors linked by some low-dimensional manifolds, offering mechanistic account of schema encoding and testable predictions for neuroscience.
(3) Dynamic analysis uncovers how attractor structures evolve during shaping: from isolated discrete attractors to integrated sequence attractors, abstract, compact attractor structure providing a novel dynamical mechanism of schema formation.
(4) Extending our sequence attractor-based shaping approach to the keyword spotting task shows improved learning efficiency, demonstrating its potential in complex, real-world tasks.
Together, our work advances the understanding of schema representation and its learning via shaping from a dynamical systems perspective, offering new insights into how abstract knowledge arises from experience through curriculum-like processes in both neuroscience and machine learning.
2 Experimental Background
To model schema learning under shaping conditions, employ the odor-sequence task of schema evolution from Zhou et al. 4 a study designed to investigate this process in rats using a typical shaping protocol. In each task instance (termed "problem"), 16 novel odors are randomly sampled and organized into two pairs of sequences (Sla - Slb and S2a S2b), shown in Fig. [i. In each of the six positions within a sequence, single odor is presented, and the rat must decide if it signals reward. Transitions among the sequences Sla, S1b, S2a, and S2b are selected at random. Although odor identities change in each new problem, the core sequence and reward structures remain the same. Thus, rats need to learn this invariant task structure for rapid adaptation. Zhou et al. adopted shaping paradigm train rats learn the challenging task. (I) Task primitive learning: Initial training on a basic odor-reward association task with 16 randomly sampled odors (Fig. [p.. (2) Task sequence learning: Introduction of structured sequence task using the same 16 odors and the predefined template (Fig. [Td), requiring the rat to link learned associations into temporal sequences. (3) Task schema learning: Introduction of five new problems successively, each with novel set of 16 odors following the same task structure (Fig. [b). The main findings of Zhou et al. are twofold: First, rats exhibit increasingly faster learning of new problems with more experience. Second, population activity in the OFC becomes progressively lower-dimensional and forms increasingly structured patterns that reflect the underlying task schema. However, the precise neural representation of tasks and the dynamics of their evolution through shaping are still unclear.
schemas for
3 Methods
3.1 Model Structure
We employ an RNN model to demonstrate schema learning via shaping on the odor-sequence task. The RNN comprises three components: an input layer with M units, recurrent layer with N units,
3

---

and an output layer with K units, as illustrated in Fig. D.. At time t, given an input stimulus vector Im (t), the total ipput x, recurrent unit n evolves according to the following dynamics:
dr,(1) (t) + dt
W (1))
W I (t),
(1)
here, Win and Wree represent the input-to-recurren and recurrent-to-recures weight matrices, respectively. The activation of the unit is computed as r, (() = ((nh(zn())), and denotes the time constant.
The output (t) of unit & is given by:
N (t)()= Σ
Wen I' (t).
(2)
The output layer linearly reads out the RNN's activity to generate task-relevant predictions, including odor classification and reward prediction, depending on the shaping stage. The recurrent-to-outppt weight is denoted by Wout, The RNN is simulated using Euler's 's method of numerical integration.
3.2 Task Design and Shaping Procedure
In the odor sequence task, each odor stimulus is represented as one-hot vector of dimension M 96, with the non-zero entry set to 20 (see Appendix Sec.A for details). Each stimulus presentation lasts T, steps. Following the shaping paradigm of Zhou et al. 4., the RNNs undergo training across three stages, detailed below.
Task Primitive Learning. In this stage, RNNs are trained on a simple prediction task that learn to associate individual odors with corresponding reward outcomes. Each 15-step trial includes a 5-step delay, 5-step odor stimulus, and a 5-step delay, and is corrupted by Gaussian white noise with a mean of and a variance of 1. The OFC flexibly encodes task-relevant variables, including odor rewards and identities [16] [77.. As both required in this task, the model's output layer comprises K = 18 units: a 2-dimensional reward prediction vector greward (t) and a 16-dimensional odor classification vector gelass (t). For rewarded odors, groward (t) = [2, O): for non-rewarded odors, greward (t) = [0, 2]. Odor identity encoded as a one-hot vector in gelass (f). Output units are required to be at zero before the odor stimulus appears, transition to target values during presentation, and remain there until it ends (I)..Sl). This target sequence mimics the ramping dynamics observed in cortical circuits during decision-making [88. Finally, the loss function used during task primitive learning is defined as regression loss:
T Σ (1v02(()) - groward (1)) + Σ (1)xx(() golass (1)]
T
(
2
The total loss is balanced between reward prediction and odor identity classification by a weighting factor 3, which we set to 0.5 here.
Task Sequence Learning. During this stage, RNNs, having been pretrained in the task primitive learning phase, undergo further training in odor-sequence problem. As described in Sec. and Fig. [d. the same 16 odors from the previous stage are organized into four sequences. Taking STa as an example, each 90-step sequence trial comprises six odor stimuli presented intervals sampled from a uniform distribution between 5 and 10 steps, mimicking experimental variability [4] To simulate sensory noise, the entire input stream is corrupted with Gaussian noise (mean variance = 1). As odor identities in the sequence task become unreliable predictors of reward (illustrated by odor 13 indicating different rewards at P4 and P5), the network shifts its prediction target only reward outcomes. Consequently, the output layer is simplified to K = 2 units, representing only the reward prediction vector greward (t), with the same values as in the task primitive learning setup. At the start of the sequence, all output targets are set to zero. Upon stimulus presentation, the outputs transition their target values and hold them until the next odor appears, at which point they update accordingly. The loss function is defined as:
T |1((() - 5 grwwadd (1)1
L2
2
1=1

---

Task Schema Learning. As shown in Fig. De. the pretrained RNN from the previous stage is further trained on a series of seou-neeuence problems sharing the same underlying structure. For each task, 16 new one-hot vectors with the active entry set to 20, different from those in the previous task, are selected from the 96-dimensional odor space, All other task parameters, including sequence duration and background noise level, remain consistent with those in task sequence learning. Thee network continues to predict only reward outcomes, with the loss defined as:
T Σ llv(() - grooward (D)12
La
(S)
Across all three stages, we use the Adam optimizer. See Appendix Sec.A for more training details.
Results
4.1 Learning-to-learn and Structured Task Representation Emerge in Shaping-Traiie Trained RNNs
a
Inputs
Recumant neural
b
No of trails to crtterion
1000
Experion: Ft
Dim
Reward prediciin
800
Dim 96
600
Dm
Dim
Class prodiction
400
Conly simple
Odor Space
Problem
Data
d
Modal
e
P1
Carrelation coefficient
Wilrouu Shaping Shapre
P2
Triel Ippe
0.9
P9
Thtl
P5
40
0.8
P0
P6
Trial type
0.7
RNN
Trial type
Figure 2: Behavior and representation in RNNs via shaping. (a) RNN architecture. A three-layer network (input, recurrent, output). The input is a 96-dimensional odor stimulus vector. The output layer predicts reward and odor class. (b) During task schema learning, the number of training trials for the shaped RNN to reach a performance criterion across consecutively presented problems (10 seeds per problem). The dashed line represents the polynomial fit. (c-d) Dissimilarity matrices. Representational dissimilarity matrices (RDM) of OFC ensembles(c) 4 and RNN hidden activities (d). Each matrix shows the distance between the neural/model representations of all pairs of trial types (24 total, 6 per sequence) across sequences and positions. (e) Pearson correlation between the experimental and model RDMs. Blue indicates RNNs trained with shaping, and pink indicates those without. Results are averaged over 10 seeds. See Appendix Sec.A for detailed parameters.
Schemas are proposed to encode abstract, generalizable knowledge, enabling faster learning in novel situations 2 []. To demonstrate this, we train an RNN using the shaping process described in Sec. 3.2 The RNN first learns a simple odor-reward association task until the training loss falls below 0.05, followed by an odor-sequence task using the same odors, again trained until the loss falls below 0.05. It is then exposed to series of novel tasks that share the same task structure but involve entirely new stimuli. Importantly, no explicit meta-learning objective is used. During this task schema learning phase, the RNN exhibits learning-to-le -o-larr behavior: the number of trials required to reach the criterion decreases significantly across problems (Fig 2p). An exponential fit reveals clear trend of accelerated learning, suggesting the RNN utilizes prior knowledge for more efficient adaptation. This pattern mirrors learning- -o-learn behavior observed in rats performing the odor-sequence task.
hallmark of schema learning observed in rats' OFC the emergence of structured neural represen- tations 4 as shown in 2 To assess this in our model, we replicate the experimental analysis by computing pairwise distances between trial types the RNN's hidden state space across positions and sequences, yielding an 24 24 RDM (see Appendix Sec. C for details). This RDM reflects the
5

---

structured geometry of the task representation space. As shown in Fig 2.d, the RNN's RDM closely aligns with that observed the OFC. Quantitatively. the Pearson correlation between the model and the experimental RDM exceeds 90% (Fig 2). indicating that the RNN captures the hierarchical task representation structure and exhibits a representational geometry similar to OFC (see Fig.S2). To test the role of shaping, we conduct an additional control experiment in which the RNN is trained directly on a set of odor-sequence problems without shaping. This control model exhibits markedly lower correlation with OFC ensembles than shaped RNNs, indicating that shaping is important for the emergence of structured task representation in the RNN.
4.2 Low-Dimensional Sequence Attractors Serve as Schema for Generalization
a 1.0
Model
Dim. Red. Indax
Dm. index
0.5
0.8
0.80
P1
Photo
Probf
P2
0.0
0
5
10
0.75
20
Proto
100
0.6
5
10
15
PCs
Epoch
Day
d
$2a
Imag(A)
PC3
PC2
0.0
1.0
0
Real(A)
Lig
MaxR(a)
1.0
fPc2
12
24
PC1
PC1
Attractors Index
Figure 3: Dimensionality and Attractor Dynamics in Shaped RNN. (a) Cumulative variance of RNN activity explained by principal components (PCs). Dashed line marks the top three PCs.(b) Di- mensionality compression over time in the RNN across five odor-sequence problems. Dim.red.index, dimensionality reduction index- is the normalized variance explained by the top three PCs. Shaded areas indicate the standard error of the mean averaged across 20 seeds. (c) Dimensionality compres- sion over time in rat OFC across two odor- sequence problems (d) PCA visualization of neural trajectories for four sequences (S1a, S1b. S2a, S2b) in the shaped RNN, projected onto the top three principal components. Six fixed points per sequence were identified via optimization-bnt
analysis. (e) Zoom-in of the shaded area in (d). Black arrow: empirical transition direction at the odor 12 fixed point. Green arrow: principal eigenvector; gray arrows: other eigenvectors of the Jacobian matrix at Eigenvalue spectrum of the Jacobian at the odor 12 fixed point. Bottom: Maximum eigenvalues across all 24 fixed points, with the red dot indicating the one corresponding to odor 12. See Appendix Sec.A for detailed parameters.
odor 12. (1) Top:
Our brains can reduce the dimensionality of neural representations during schema formation by averaging unique problem details while preserving cross pooss-problmm commonalities (3) To examine this in our RNN, we perform PCA on hidden activities during task schema learning. Fig. 33 shows the network dynamics reside in low-dimensional subspace, with the top three PCs explaining nearly 80% of the variance. We further track dimensionality change over learning. Fig. 3 shows that the explained variance of the top 3 PCs progressively increases within and across problems, indicating dimensionality compression neural activity over learning, similar to OFC activity in rats (Fig. B
Together, these findings suggest that a low-dimensional schema code emerges in RNNs trained via shaping, evident in both behavior and representation. However, the underlying neural dynamics of this code remain unclear. To investigate this, we visualize the RNN's state dynamics using PCA. As shown in Fig and Fig.S3, the four sequences (Sla, S1b, S2a, and S2b) form distinct low- dimensional trajectories in the RNN state space. Using optimization peimizztioo-based fixed point analysis [15], we identify six fixed points along each trajectory, each fixed point corresponding to a specific odor's position within the sequence (detail methods see Appendix Sec.C.2). Sequences sharing similar reward and transition structures (e.g., Sla, S1b, S2a) have close trajectories, indicating abstraction of representations with structural commonalities. To probe the stability of these fixed points,

---

examine the eigenvalues of the local Jacobian matrix (Fig.BD. All maximal eigenvalues are below indicating that the fixed points are locally stable and act as attractors. Notably, the principal eigenvector associated with the largest eigenvalue aligns with the direction of sequential transitions (Fig.Br 3p, suggesting mediates movement along the trajectory.
These results reveal that the RNN encodes odor sequences as sequence attractors sequences of stable fixed points connected by low-dimensional manifolds. Through perturbation study (Fig.S4), we further demonstrate that this sequence attractor schema serves as reusable flexible template, enabling new odor cues to bind and thereby facilitating efficient generalization in novel situations.
4.3 Gradual Emergence of Sequence Attractor Schema Through Shaping
a
0.5
C
600
0.0
No of Saatsss ortente
Whitosut Shappng
0..9
Shapping
3.0
Real())
1.0
#00
MedR(A)
200
Allratoors Index
RNN
GRU
L5TM
S1a
Distance
375
F2B
PC2
Epoch
100
200
g
h
Distance
POA,
-10
-5
10
20
PG2
Problem
Figure 4: Evolution of attractor dynamics in RNNs during shaping. (a-c) c) Task primitive learning. (a) PCA visualization of identified fixed points; red rewarded states, blue non-rewarded. (b) Stability analysis of fixed points: top, eigenvalue spectrum of a sample odor 8; bottom, maximal eigenvalues across fixed points. (c) Learning efficiency in the subsequent sequence learning stage, comparing models with and without prior task primitive learning, averaged over 5 seeds. Note that the total trial number criterion includes both task primitive and sequence learning stages for fair comparison. (d f) Task sequence learning. (d) PCA visualization of attractor dynamics reveals the gradual shift of the discrete attractor for odor (primitive learning) toward its position in the sequence attractor associated with Sla. Green curve: neural trajectory of Sla. Solid circle: prior attractor from task primitive learning; hollow circles: attractors of Sla sequence. (e) Zoom-in on the learning trajectory of the odor 1 attractor. (f) The average Euclidean distance between evolving attractor states and their corresponding locations of sequence attractors (task sequence learning) over training epochs (averaged over 5 seeds). (g -i) Task schema learning. (g) PCA visualization of attractor abstraction. Green curve: converged abstract neural trajectory of Sla, Slb, and S2a. Numbered hollow circles: prior attractors (sequence learning). Unnumbered hollow circles: attractors of the abstract trajectory (task schema learning). (h) Zoom-in of the abstraction process (g), illustrating the migration of prior attractors: odor 0 at Sla, odor 2 at S1b, and odor 8 at S2a (sequence learning), toward the abstract attractor within the final sequence attractors (schema learning).
(i) The average Euclidean distance between attractors of corresponding sequence steps in Sla, S1b, and S2a, tracked during problem training (averaged over seeds). See Appendix Sec.A for detailed parameters.
Given that sequence attractors serve as schema, we next investigate their evolution during shaping. We analyze attractor dynamics across three shaping stages. First, during task primitive learning (Fig.pp. b), the RNN learns simple odor-reward associations and develops 16 discrete, locally stable attractors, each representing a specific odor cue. Importantly, RNNs initialized with these prior discrete attractors learn the subsequent task sequence significantly faster, requiring fewer trials to
7

---

reach criterion (Fig. 4b). This learning benefit is also consistent across diverse recurrent architectures, such as LSTM and GRU.
During task sequence learning, we track how prior attractor states in the RNN's state space evolve and examine whether they't re reused to form sequence attractors, shown in Fig. [h. At each training step, we probe the RNN's attractor states. Taking odor I as an example, we initialize the hidden state to its prior attractor and iterate the network dynamics without input until convergence new stable state. This new state is then used as the initial state for the next training batch iteration, repeating this process to generate the evolving trajectory of stable states. Fig. Aie illustrate the gradual migration of odor I's prior attractor (primitive learning) to its corresponding location within the Sla sequence attractor structure (sequence learning). We found that 87% (14 out of 16) of prior attractors successfully evolve to their correct target locations. Quantitatively. the average distance between these successfully evolving states and their corresponding attractor locations (sequence learning) steadily decreases over training (Fig. I detail methods see Appendix Sec.C.3). These findings suggest that attractors learned during task primitive learning are reused, reorganized, and linked to form sequence attractors during sequence learning.
During task schema learning, we examine how sequence-specific attractors are compressed into abstract sequence attractors. FiguresFg..t illustrate this process: the attractors for odor in Sla, odor 2 in Slb, and odor in S2a gradually converge toward a common attractor. The average distance between attractors occupying the same sequence position across Sla, S1b, and S2a also decreases over training (Fig. T.. These results suggest that as the network learns related problems presented consecutively, it gradually averages out sequence-speciff details while retaining commonalities shared across S1a, a, S1b and S2a eventually forming a unified abstract sequence attractor. This convergence is expected, given the common reward transition structure across these sequences.
Together, these findings reveal a dynamic, attractor-based mechanism for schema formation through shaping. First, discrete attractors emerge during task primitive learning. These are then organized into sequence-specific structures during task sequence learning. Finally, in task schema learning, these structures are compressed and abstracted into unified schema. This sequence attractor-baa based schema shaping process supports our conceptual idea shown in Fig. Ip.
4.4 Divergent Learning Dynamics in RNNs with and without Shaping
1.0
Whhu win
Dim Red Indox
30
waan Shepre
Wins Sheping
Accuracy
Distames
20
0.5
0.65
10
0.0
P4
P5
0,75
20
100
2
Posttion
Problem
13
PC3 >
14
b01
PQ2
PC1
PG1
Figure 5: Evolution of attractor dynamics in RNNs without shaping. (a) Performance comparison of RNNs with and without shaping at P4 and P5 in S2a/S2b. Networks were trained and evaluated after problem I learning (averaged over 5 seeds). (b) Dimensionality compression dynamics in RNNs without shaping across five odor-sequence problems (averaged over 5 seeds). Other settings are iden- tical to Fig. Fig.Bp. (c) Average Euclidean distance between the evolving attractor states corresponding to P4 and P5 across S2a and S2b sequences for RNNs with and without shaping during problem training (averaged over 5 seeds). (d) PCA visualization of sequence attractors
representative RNN without shaping. Sequence attractors in S2a and S2b converge into single trajectory after problem 1 learning.
(e) Zoomed-in view of attractor states for P4 and P5 in S2a and S2b. (f) PCA visualization of attractor states at the same corresponding positions in problem 5 as shown in (e).
8

---

To elucidate the mechanisms underlying behavioral differences between shaped and unshaped RNNs, we examine the learning dynamics of unshaped
Networks.Shapee RNNs achieve markedly higher reward prediction accuracy (99.89 + 0. 0.03%) than unshaped ones (93.20 + 2.4%), with the largest discrepancy in classifying odors at P4 and P5 of S2a/S2b, where reward contingencies are reversed. As shown in Fig. 5S. shaped RNNs reach near-perfect accuracy, whereas unshaped RNNs reach only 25% after problem 1. Applying the same PCA-based dimensionality analysis as in Fig. 3 (Fig. 5p), we find that the dimensionality reduction in unshaped RNNs differ qualitatively: (I) Premature compression - unshaped networks enter a low-dimensional regime (93%) much earlier than shaped ones (82.5%), indicating early collapse coarse representations; and (2) Lack of progressive abstraction unlike shaped networks, unshaped ones show little change in dimensionality across problems. These findings reveal fundamentally distinct learning dynamics between shaped and unshaped RNNs.
We further analyze and visualize sequence attractors to better understand representational evolution. In shaped networks, the primiitimiiee learning stage already establishes four distinct attractors for the odor events at P4 and P5 (in S2a and S2b). These attractors serve as stable building blocks for higher-level, sequence-specific attractor trajectories in later stages, maintaining a consistent distance between corresponding attractor states (Fig. ()). In contrast, unshaped networks lack this scaffold. Early in training. they represent P4 and P5 odors in S2a and S2b with overlapping trajectories, and solving the task therefore requires the later formation of separate attractor trajectories a process that is often unstable and leads to failed convergence. Fig. [ shows a successful example of late attractor trajectory separation, where the trajectory for problem 5 diverges from the shared trajectory established after learning problem 1.
In summary, shaped networks build schemas progressively using scaffolded attractor dynamics and structured compression, whereas unshaped networks exhibit abrupt, shallow compression that leads to poor generalization. These findings demonstrate that shaping fundamentally alters the learning trajectory and attractor organization of RNNs, clarifying how and why shaping facilitates learning.
4.5 Sequence Attractor-Bant Based Shaping Improves Learning Efficiency in Keyword Spotting
a
b
55
No. of epach to criterion
Without Sheping
PC3;
With Shaping
40
X
y-h
w--a
25
PC2
'en-dol
"ax''
'sn' 'dx-exr''
had [hv [en-de]] year [yxr)
['w-ao'; 'sh'l water [waao,, 'dx-axxr)
10
GRU
LSTM
PC1
Figure 6: Sequence attractor-based shaping in a keyword spotting task. (a) The task involves four keywords: "water," "wash," "had," and "year." The upper panel shows the MFCC representation of a sample utterance of "watrr," while the lower panel illustrates the tree-structured phoneme organization of keywords. (b) Learning efficiency comparison (epochs to 90% accuracy) between models trained with and without shaping, averaged over 5 seeds. (c) PCA visualization of RNN neural trajectories after shaping. Crosses indicate identified fixed points. See Appendix Sec for detailed parameters.
Based on the sequence attractor-based shaping framework, we apply our method to a woaldworld sequence task: keyword spotting (Task details see Appendix Sec.D.1). The task involves four spoken words - "water," "wash,"" "year," and "had"- each composed of basic phonemes forming a hicrarchical structure (Fig. (a)). These words are randomly selected from the TIMIT dataset [1)]. Following the shaping approach, we divide the task into three stages. In task primitive learning. the model learns to classify phonemes using only one sample per word, forming discrete phoneme-leve attractors. In task sequence learning, the model links these attractors by training on sampled word sequences for word class prediction. Finally, in task schema learning. the model trains on the full dataset to abstract schema and generalize across word variations.
As shown in Fig shaping significantly improves learning efficiency compared to a model trained without shaping, requiring fewer epochs to reach 90% test accuracy. PCA visualization of hidden

---

states reveals a tree-like structure consistent with the phoneme hierarchy, supporting the emergence of a sequence schema (Fig/Op) Additionally. we identify seven leaf attractors corresponding to individual phonemes, and one root attractor representing the initial state prior to phoneme onset. These results demonstrate that the sequence attractor-based shaping framework generalizes to complex sequence learning tasks, offering both interpretability and efficiency.
5
Discussion
Sequence schemas form the core of flexible and generalizable intelligence across animals and humans. For complex tasks, acquiring such schemas through pure trial-and-error is often ineffective. Instead, animals and humans use shaping breaking down complex tasks into simpler subtasks and gradually assembling the full schema. However, the underlying dynamic mechanism behind this process remains unclear. In this work, we use RNNs to study how schemas are represented and evolve through shaping. Our main findings are fourfold: (1) We systematically replicate key behavioral and neural features of schema learning observed in the OFC of rats. (2) We show that sequence schemas can be encoded as sequence attractors. (3) We identify novel dynamic process of schema formation, progressing from point attractors to sequence attractors, and finally to abstract schemas. (4) We demonstrate the practical utility of this framework by successfully applying it to keyword spotting. These findings may advance understanding of the neural mechanisms of schema learning via shaping and offer new insights into how abstract, generalizable structures can be learned from experience in both biological and artificial systems.
Related Works Schema learning has attracted increasing attention in neuroscience 3
15
20
22 23 27 Previous studies have primarily examined schema representations for example, low- dimensional manifolds attractors in sensorimotor 13, decision-making tasks [15) or capturing temporal order [20]. However, a fundamental question remains overlooked: how does the brain learn schemas through shaping? Despite its prevalence in animal training. the mechanism of shaping is systematically underexplored in neuroscience. And previous works mainly focus on behavioral outcomes, neglecting the underlying dynamics of abstract knowledge acquisition [2] 22] Our work addresses this deficit by investi gating schema learning via shaping, providing the first explicit link between these two essential concepts.
Our approach mirrors curriculum learning by decomposing complex tasks into simpler subtasks However, while curriculum learning in machine learning primarily focuses on optimizing performance generalization 25 26), it typically lacks a mechanistic understanding of how the underlying abstract knowledge is acquired during the process (25). In contrast, we offer a dynamic framework grounded in biological learning: learning progresses from primitive attractor structures to reusable abstract attractor schemas that facilitate generalization (15). This framework may provide inspiration for new curriculum learning algorithms that explicitly facilitate the learning and use of abstract knowledge.
to boost learning efficiency 24 (25
Recent studies have questioned the identifiability and robustness of RNNs in neuroscience modeling, showing they can achieve similar behaviors via distinct internal solutions (28). Our analysis, however, reveals that shaped RNNs converge to highly similar internal dynamics regardless of initialization. Applying Canonical Correlation Analysis (CCA) to models trained with different seeds (Fig.S6), find strong alignment among the top three canonical components, indicating consistent representation geometry across models. Previous findings show that task complexity or multi-task learning can reduce degeneracy in RNN solutions (28 291. we extend this by demonstrating that our shaping paradigm may be an effective way to reduce this degeneracy.
Limitations and Future Works. A limitation of this work is that model optimization relies on backpropagation through time, which is not biologically plausible [30]. Future work could explore combining dopamine- based reward learning [I with slow Hebbian plasticity [32 to develop more biologically grounded mechanisms for schema learning. Another limitation is that model includes only the PFC, whereas schema formation in the brain is thought to result from dynamic interactions between the PFC and the hippocampus [17] 33 34), which play complementary roles with distinet learning timescales [35]. The hippocampus may support task primitives and sequence learning, while the PFC contributes to task schema learning. Modeling their interaction could help clarify the neural basis of schema formation through shaping and inspire more flexible and generalizable sequence learning algorithms.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Science and Technology Innovation 2030 Major Program (No. 20212D0203700) 20212D0203705, Y.Y. Mi), National Natural Science Foundation of China (62336007, B. Hong).
References
[1] Gilboa, A., & Marlatte, H. (2017). Neurobiology of schemas and schema-mediated memory. Trends in cognitive sciences, 21(8), 618-631.
[2] Tse, D., Langston, R. F., Kakeyama, M., Bethus, I, Spooner, P. A., Wood, E. R.,.. & Morris, R. G. (2007). Schemas and memory consolidation. Science, 316(5821), 76-82.
[3] Bein, O.. & Niv. Y. (2025). Schemas, reinforcement learning and the medial prefrontal cortex, Nature Reviews Neuroscience, 1-17.
[4] Zhou, J., Jia, C., Montesinos-Carteee Cartagena, M., Gardner, M. P. Zong, W., & Schoenbaum, G. (2021). Evolving schema representations in orbitofrontal ensembles during learning. Nature, 590(7847), 606-611.
[5] Harlow, H. F. (1949). The formation of learning sets. Psychological review, 56(1), S1.
[6] Ma. F., Lin, H., & Zhou, J. (2025). Prediction, inference, and generalization in orbitofrontal cortex. Current Biology, 35(7), R266-R272. R272.
[7] Maor, I,, Atwell, J., Ascher, I., Zhao, Y., Takahashi, K., Hart, E.. ... & Schoenbaum, G. (2025). Persistent representation of prior schema in the orbitofrontal cortex facilitates learning of conflicting schema. bioRxiv, 2025-02.
[8] Makino, H. (2023), Arithmetic value representation for hicrarchical behavior composition. Nature neuro- science, 26(1), 140-149.
[9] Mi, Q., & Summerfield, C. (2025). Human curriculum learning of cue combination task.
[10] Lec, J. H., Mannelli, S. S., & Saxe, A. (2024). Why do animals need shaping? a theory of task composition and curriculum learning. arXiv preprint arXiv:2402. 18361.
[11] Mante, V., Sussillo, D., Shenoy, K. V., Newsome, T. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. nature, 503(7474), 78-84.
[12] Inagaki, H. K., Fontolan, L., Romani, S., Svoboda, K. (2019). Discrete attractor dynamics underlies persistent activity in the frontal cortex. Nature, 566(7743), 212-217.
[13] Goudar, V., Poysakhovich, B., Freedman, D. J., Buffalo, A., & Wang. X. J. (2023). Schema formation in neural population subspace underlies learning-to-leann in flexible sensorimotor problem-solving Nature Neuroscience, 26(5), 879-890.
[14] Yang, G. R., Joglekar, M. R., Song, H. F,, Newsome, W. T., & Wang, X. J. (2019). Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2). 297-306.
[15] Driscoll, L. N., Shenoy, K., & Sussillo, D. (2024). Flexible multitask computation in recurrent networks utilizes shared dynamical motifs. Nature Neuroscience, 27(7), 1349-1363.
[16] Schoenbaum, G., & Eichenbaum, H. (1995). Information coding in the rodent prefrontal cortex. I. Single- neuron activity in orbitofrontal cortex compared with that in pyriform cortex. Journal of neurophysiology. 74(2). 733-750.
[17] Wikenheiser, A. M.. & Schoenbaum, G. (2016). Over the river. through the woods: cognitive maps in the hippocampus and orbitofrontal cortex. Nature Reviews Neuroscience, 17(8). 513-523.
[18] Wang. X. J. (2008). Decision making in recurrent neuronal circuits. Neuron, 60(2), 215-234,
[19] Garofolo, S., Lamel, L. F., Fisher, W. M., Fiscus, J. G.. Pallett, D. S. Dahlgren, N. L. (1993). DARPA TIMIT Acoustic Phonetic Continuous Speech Corpus CDROM NIST
[20] Zou, X., Chu, Z., Guo, Q. Cheng, J.. Ho, B., Wu, S., & Mi, Y. (2023). Learning and processing the ordinal information of temporal sequences in recurrent neural circuits. Advances in Neural Information Processing Systems, 36, 34020.

---

[21] Krueger, K. A., & Dayan. P. (2009). Flexible shaping: How learning in small steps helps. Cognition. 110(3), 380-394.
[22] Hocker, D., Constantinople, C. M., & Savin, C. (2024). Compositional pretraining improves computational efficiency and matches animal behavior complex tasks. bioRxiv.
[23] Bruch, S., McClure, P., Zhou, J., Schoenbaum, G., & Pereira, F. (2021). Validating the Representational Space of Deep Reinforcement Leaminng Models of Behavior with Neural Data. bioRxiv, 2021-06.
[24]
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009). Curriculum learning. In Proceedings of the 26th annual international conference machine learning (pp. 41-48).
[25] Wang, X., Chen, Y., & Zhu, W. (2021). A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence, 44(9), 4555-4576.
[26] Bauer, J., Baumli, K., Behbahani, F., Bhoopchand, A., Bradley-Schmiee N., Chang, M., & Zhang, L. M. (2023). Human-timescale adaptation in open-ended task space. In International Conference Machine Learning (pp. 1887-1935). PMLR.
[27] Khona, M., & Ficte, I. R. (2022). Attractor and integrator networks in the brain. Nature Reviews Neuro- science, 23(12). 744-766.
[28] Huang, A., Singh, S. H., Martinelli, F,, & Rajan, K. (2025). Measuring and controlling solution degeneracy across task-trained recurrent neural networks. ArXiv, arXiv-2410.
[29] Yang, G. R., Cole, M. W., & Rajan, K. (2019). How to study the neural mechanisms of multiple tasks. Current opinion in behavioral sciences, 29, 134-143.
[30] Lillicrap, T. P., Santoro, A., Marris, L.., Akerman, J. & Hinton, G. (2020). Backpropagation and the brain, Nature Reviews Neuroscience, 21(6), 335-346.
[31] Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. & Botvinick, M. (2018). Prefrontal cortex as a meta-reinforreemet learning system. Nature neuroscience, 21(6), 860-868.
[32] Hattori, R., Hedrick, N. G., Jain, A., Chen, S., You, H., Hattori, M., ... & Komiyama, T. (2023). Meta- reinforcement learning via orbitofrontal cortex. Nature Neuroscience, 26(12), 2182-2191.
[33] Lin, H., Zhou, (2024), Hippocampal and orbitofrontal neurons contribute to complementary aspects of associative structure. Nature Communications, 15(1), 5283.
[34] Zou, X., Cao, X., Yang, X., & Hong, (2024). Leveraging attractor dynamics spatial navigation for better language parsing. Forty-firs first International Conference on Machine Learning.
[35] Kumaran, D.,. Hassabis, D., & McClelland, (2016). What learning systems intelligent agents need?
Complementary learning systems theory updated. Trends in cognitive sciences, 20(7), 512-534.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the main claim, the evidence is provided Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in the section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

This paper does not include formal theoretical results proofs, as it focuses on neural dynamical modeling and empirical analysis without introducing new theorems. Guidelines:
Justification:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The model architecture, training procedures, and experimental setup are detailed in Section 3 and the technical appendices.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
14

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Code is not publicly available at submission, but Section 3 and technical appendices provide detailed instructions to reproduce the main results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Training and test details are fully described in Section and technical appen- dices.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Error bars representing the standard deviation over multiple runs with different random initializations are reported in Fig Fig Ba; 4b.fi and Fbg while the standard error of the mean (SEM) is reported in Fig.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Experiments ran on an Intel i9- 19-14900k CPU with 32GB RAM, no GPU used. Each training took 30 minutes, with 200 runs for tuning and validation.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.. httpa//ii...ccpuulelecccccccceee
Answer: [Yes]
Justification:
Our research adheres fully the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work is foundational and theoretical, without immediate societal applica- tions impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
16

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
This work does not release any models or datasets that pose high risks of misuse.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use the TIMIT dataset (LDC93S1) with proper citation, under our institu- tion's license (LDC User Agreement for Non-Memnn Non-Meebers).
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
The paper does not introduce any new assets. It only uses existing available datasets and standard models.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The research does involve any crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not involved in the core methods.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
19


Reviews:
[{"review_id": "ZojcVDqLl9", "reviewer": "Reviewer_Qruw", "summary": "This paper presents a computational model of schema formation through shaping, inspired by neurocognitive processes. Using a curriculum learning paradigm modeled after animal studies (specifically odor-sequence tasks in rats (REF 4)), the authors train RNNs to form discrete attractors, link them into sequential dynamics, and ultimately abstract these into low-dimensional sequence attractors representing schema. The model replicates key experimental findings from orbitofrontal cortex (OFC) studies, including faster learning (learning-to-learn), dimensionality compression, and structured representation geometry. The framework is also tested on a keyword spotting task to demonstrate broader applicability. The manuscript is clearly written.", "questions": "1- From Fig. 4c, it looks like LSTM has much better performance in terms speed of learning. Can authors elaborate the reason for this? And if this is the case, why did the authors choose to work with RNNs rather than LSTMs?\n\n2- In a similar vein, do LSTMs have similar attractor dynamics when solving this task?\n\n3- I understand the timing constraint in running extra models, but can authors compare their current model and training strategy with other recent models such as meta-reinforcement learning model (Yang, Nature Neuro, 2018)?\n\n4- the analysis in Fig. 4g-i is very cool. I am wondering what is the difference of this trajectory between \"with shaping\" and \"without shaping\" networks. In other words, do both types of networks follow similar trajectories to get to abstract state or these is fundamental difference between them?", "limitations": "I agree with authors that limitations of this work are reliance on back-propagation and lack of multi area interactions. Those are good avenues for future improvements.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths: The paper introduces a novel dynamical systems account of schema formation, grounded in neuroscience and with relevance for both computational neuroscience and machine learning. The shaping paradigm is directly motivated by animal training literature and aligned with known properties of the prefrontal cortex and OFC. The RNN replicates key neural signatures observed in OFC recordings, including the progressive compression of dimensionality and structured representational dissimilarity matrices (RDMs), with high correlation to empirical data. The paper provides a compelling narrative of schema evolution: from discrete point attractors → sequence attractors → abstract attractors.  \n\nWeaknesses: \n-It's unclear how architecture-dependent the attractor formation is. For example, do gated RNNs (LSTM/GRU) or Transformers exhibit similar attractor dynamics under shaping?\n\n-Most results rely on a single neuroscience-inspired task. The keyword spotting task is a welcome addition, but further validation on more varied or large-scale datasets (e.g., navigation or language tasks) would demonstrate scalability and robustness.", "was_revised": true, "final_justification": "The authors have thoroughly addressed my concerns and plan to add extra analysis to final version of the manuscript. \n\nThis work is valuable because it is one step towards understanding the neural dynamics of schema formation which is an active area of research in systems neuroscience. We still don't know how the brain builds abstract concepts. The results of this paper contribute to topics such as compositionality and meta learning in neuroscience and ML.\n\nAdding the authors' new results on the learning trajectory of shaped and unshaped networks along with mechanistic interpretation is valuable for the revised manuscript.", "rebuttal": "We thank the reviewer for the positive assessment of our work and valuable feedback. Our point-by-point replies are below.\n\n***On Weakness:***\n1. Thank you for the helpful comment. To assess the architectural dependence of attractor formation, we conducted additional experiments using LSTM and GRU networks with 200 recurrent units, trained under the same shaping condition as the vanilla RNN.\n\n| Network\t | RDMs Correlation \n|  :--: | :--:  \n|LSTM with shaping\t | 0.89 ± 0.01  \n|LSTM without shaping\t | 0.87 ± 0.03  \n|GRU with shaping\t | 0.906 ± 0.01  \n|GRU without shaping\t | 0.87 ± 0.007\n|RNN with shaping\t | 0.902 ± 0.02  \n\nAs shown, both gated architectures achieve comparable representational similarity to neural data. We also visualized their dynamics and observed the emergence of similar discrete, sequential, and abstract attractors, indicating that our attractor-based shaping approach generalizes well across architectures\n\nRegarding Transformers, due to their feedforward nature and lack of intrinsic recurrence, they do not exhibit attractor dynamics in the same sense. Extending our approach to more network architectures is an interesting direction for future work.\nWe will include these comparative results and analyses in the revised manuscript.\n\n2. Thank you for the helpful suggestion. We agree that evaluating the framework on more diverse and large-scale tasks is important for demonstrating its scalability and robustness.  \n\nThe sequence attractor schema framework is conceptually applicable to domains such as spatial navigation, where abstract spatial relations across environments could be encoded as attractor sequences [1], and language understanding, where narrative structures may be represented as sequential event schemas [2]. These domains offer promising opportunities to test shaping-based learning in more complex settings.  \n\nWe will discuss these potential extensions in the revised manuscript to better clarify the generality and future applicability of our approach.\n\n***On Questions:***\n\n1. Thank you for the comment. LSTMs do exhibit faster learning, likely due to their gating mechanisms and memory cells, which help maintain gradient flow and capture long-range dependencies - thus improving performance. Notably, LSTMs with shaping still learn faster than those without shaping, consistent with the benefit observed in vanilla RNNs.\n\nWe chose to focus on vanilla RNNs for two key reasons:  \n(1) They are more biologically plausible and commonly used in neuroscience modeling [3,4], and  \n(2) Their simpler dynamics allow clearer analysis of attractor formation and transitions.  \nImportantly, our results show that shaping induces similar attractor structures in both RNNs and LSTMs.   \nWe will include the LSTM results and clarify RNNs' rationale in the revised manuscript.\n\n\n2. Yes, LSTMs exhibit similar attractor dynamics to those observed in vanilla RNNs when solving this task. As detailed in our response to Weakness 1, shaping induces comparable discrete, sequential, and abstract attractor structures across both architectures. We include supporting visualization results in the revised manuscript.\n\n\n3. Thank you for the thoughtful comment. Our model, like those in Yang et al. [5] and Wang et al. [6], uses recurrent neural networks, so there are no substantial architectural differences. The key distinctions lie in the training strategies and objectives.\n\nYang et al. (2019) employ multi-task training across diverse rule-based decision-making tasks to learn shared dynamic motifs or schemas. In contrast, we use a shaping-based curriculum to build sequence attractor schemas progressively. These strategies are complementary - Yang et al.'s approach could be used to learn basic attractor structures in task primitive learning, which would benefit subsequent sequence and schema learning.  \n\nWang et al. (2018) employed meta-RL to train LSTMs for acquiring abstract task knowledge or schema. However, their focus was on learning-to-learn behavior, not attractor dynamics. Moreover, meta-RL often necessitates an explicit meta-objective with an inner-outer optimization loop. In contrast, our approach yields schema-like attractors that emerge naturally from learning a series of related tasks without a explicit meta-objective, making it more biologically plausible.  \n\nWe will clarify the distinctions in the revised manuscript.\n\n\n4. Thank you for the insightful question. Networks trained with and without shaping follow fundamentally different trajectories toward abstraction—differences that manifest both in behavioral outcomes and the underlying neural dynamics.\n\n(1) **Behavioral divergence:**  Networks trained without shaping often converge to suboptimal solutions. For example, overall reward prediction accuracy is significantly higher with shaping (99.89 ± 0.03%) than without (93.20 ± 2.4%). This discrepancy is most pronounced for odors at P4 and P5 in S2a/S2b—where reward contingencies are reversed. Here, the unshaped network performs at near chance level (56.8 ± 31.01%), while the shaped network reliably resolves the contrast (98.6 ± 4.61%). This suggests that the unshaped network fails to properly resolve this critical contrast.\n\n(2) **Attractor dynamics：** We visualized hidden state trajectories to better understand how representations evolve. In shaped networks, task primitive learning stage already establishes four distinct attractors for the odor events at P4 and P5 (in S2a and S2b), which serve as stable building blocks for constructing higher-level, sequence-specific attractor trajectories in later stages. By contrast, unshaped networks lack this scaffold. Early in training, they represent P4 and P5 odors in S2a and S2b using overlapping attractor trajectories. Solving the task thus requires the network to later split these shared trajectories - a process that is often unstable and leads to inconsistent or failed convergence.\n\n(3) **Dimensionality dynamics:** We also applied PCA-based dimensionality analysis (as in Fig. 3b) to RNNs without shaping to confirming the above process. Results are summarized below:\n\n| Epochs  |  0  |  10  |  20  |  30  |  40   \n| :---: |   :----:  | :---: | :--: | :---: |  :--:\n| P1 | 0.30 $\\pm$ 0.01 |  0.94 $\\pm$ 0.003 | 0.93 $\\pm$ 0.002 |  0.93 $\\pm$ 0.003 | 0.93 $\\pm$ 0.002\n| P2 | 0.90 $\\pm$ 0.04 |  0.92 $\\pm$ 0.005 | 0.93 $\\pm$ 0.006 | 0.93 $\\pm$ 0.004  |0.93 $\\pm$ 0.003\n| P3 |   0.77 $\\pm$ 0.037 | 0.92 $\\pm$ 0.005 | 0.94 $\\pm$ 0.003  |0.94 $\\pm$ 0.003 | 0.94 $\\pm$ 0.002\n\nWhile both models show dimensionality reduction with learning, the nature of this compression differs:  \n•\tPremature compression: Unshaped networks reach a low-dimensional regime (93%) much earlier than shaped ones (82.5% at the same Problem 1 stage), suggesting early collapse into coarse representations.  \n•\tLack of progressive abstraction: Unlike shaped networks, which show progressive compression from P1 to P3—reflecting averaging problem-specific features and preserving shared structure [7] —unshaped networks show minimal change across tasks. This indicates poor refinement and suboptimal optimization.\n\n**Summary:** In shaped networks, abstraction emerges gradually through scaffolded attractor dynamics and structured compression. Conversely, in unshaped networks, the compression is abrupt, shallow, resulting in poor generalization. These findings indicate that networks with shaping and those without follow substantially different learning trajectories. We will include these results and visualizations in the revised manuscript to further clarify the impact of shaping on learning dynamics.\n\n\nReferences:\n\n[1] Farzanfar et al., Nature Reviews Neuroscience, 2023   \n[2] Baldassano et al., Journal of Neuroscience, 2018   \n[3] Driscoll et al.,Nature Neuroscience,2024   \n[4] Goudar et al.,Nature Neuroscience,2023  \n[5] Guangyu Robert Yang et al., Nature Neuroscience, 2019   \n[6] Jane X. Wang et al., Nature Neuroscience,2018   \n[7] Oded Bein et al., nature reviews neuroscience,2024"}, {"review_id": "zaKtMWUztK", "reviewer": "Reviewer_hM6o", "summary": "Inspired by rat study on odor-sequence task (Zhou et al), the authors model schema learning using RNNs trained with shaping. They describe a 3-stage learning process: first, learning basic task primitives (classification of odor identity and reward); then combining them into sequences (reward only); and finally training on a generalized version of the task (reward only). The trained RNNs show hidden state representations similar to the neural activity in rat OFC. This is shown by RDM analysis, and progressive dimensionality compression (PCA). They also reverse-engineer the RNN, showing it first develops discrete attractors associated with each odor, which are later reused to form sequences. Finally, they generalized the 3-stage shaping learning process to additional experiments of keyword spotting.", "questions": "1.\tIn the Zhou et al paper, the first paragraph highlights the special role of odors 13-14. It’s very easy to miss this detail in the current paper. I think it’s important to emphasize it when describing the task.\n2.\tYou state that “Zhou et al. found it very challenging to train rats to learn the task directly, so they adopted a shaping approach”. In the Zhou et al paper, they describe shaping in Methods, but don’t mention anything about challenges. \n3.\tUnlike rats, RNNs are easily manipulable, so it’s unclear why the authors didn’t leverage this to better understand why shaping is so effective or design improved shaping strategies that could be tested in animal experiments. For example, could training in an interleaved fashion—alternating between L1 and L2—speed up learning?\n4.\tFrom my understanding, all weights are updated throughout the 3 training stages. How would the results change if certain layers were frozen and not modified across stages?\n5.\tWhat is the overall performance of the control model? How does the loss curve of this model compare to the shaping model? The RDM matrix is less similar, but it would still be useful to see it (and the dendrogram of Figure S2).\n6.\tCould the reduced similarity between the control model and the empirical rat data be due to the P4–P5 odors (13–14) which are reversed and not trained for identification as in the shaping RNN model? Also, it’s unclear why this task choice was made—some details seem to be missing (I’m aware the original study did the same reversing).\n7.\tWhy use 20 for the non-zero entry and 2 for the desired output?\n8.\tWhy is learning rate in schema an order of magnitude slower than task primitive?\n9.\tThe Zhou experiment always moves from S1 to S2 to S1 and so on. Why not do the same in simulations?\n10.\tThe difficult odor pair 13-14 is treated differently in the shaping in the Zhou experiment. Training is first done on S1 odors, then on 13-14 (including reversals), and then on the rest of S2. Why not do the same in simulations?\n11.\tWhy include odor output in the first stage and not in the second one? Behavior doesn’t require odor identity in either stage. If the OFC always outputs identity, it should continue doing so in all stages.\n12.\tFigure 2B. The legend “experiment fit” suggests that this is data from the Zhou experiment. If I’m not mistaken, this is just a fit to the simulations. Also – does this fit contribute to understanding the results? Also – the caption says polynomial, and the main text exponential.\n13.\tLine 168: “This pattern mirrors learning-to-learn behavior observed in rats performing the odor-sequence task.”. It is not that easy to find this in the Zhou paper. In fact, extended figure 1 in the Zhou paper does not appear to show any evidence of faster learning. Figure 5 in that paper is the closest I could find, which shows changes the rate of changes in latency – but not as a function of problems.\n14.\tSupp Line 33 “Each major branch is then further divided into six sub-branches according to spatial position.” First, there are less than 6 options in each major branch. Second, I don’t see this in Figure S2. \n15.\tFigure S4: Why is smaller learning rate for “Ours” a fair comparison? The measure is number of trials.\n16.\tFigure S6 panel C. Typo. Cross – corss \n17.\tThe results of Figure S7 are intriguing, suggest new experiments, and perhaps should be in the main text.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths \n The paper addresses a very interesting topic, shaping in training, which is indeed critical to both animal and artificial learning. In practice, certain curricula/ shaping are often essential to achieve high behavioral accuracy. \nThe resemblance between the RNN and empirical data seems convincing.\nThe effect of curriculum on emergence of dynamical objects has been studied, but not extensively – so this is a very welcome contribution.\n\n\nWeakness\n1.\tThe paper focuses more on replicating empirical observations of Zhou et al than on exploring why shaping is necessary. While similarities between RNNs and neural representations in animals are well documented in past, the mechanisms that make shaping essential remain unclear—an open question that could significantly advance the field.\n2.\tI'm not fully convinced that only the specific 3-stage training procedure can produce the observed similarity between RNNs and rats. The reuse of attractors has been described in previous work (Driscoll et al Nature neuro 2024, Turner et al Neurips 2023 ). This raises the question of whether similar results would emerge under other training protocols not explicitly following the 3-stage design (Control conditions are missing).\n3.\tThe final experiments, intended to demonstrate generalization of the 3-stage shaping approach, seem too similar to the initial task. It’s unclear what truly distinguishes them or how much generalization is actually being tested.\n4.\tSome training details are different from those of the experiment, and many details seem arbitrary. This raises the question of how robust are the results to these specific choices. Questions below detail these issues.", "was_revised": true, "final_justification": "I am maintaining my score of 4, but willing to engage in discussion with reviewers/AC on whether to increase to 5.\nI think this is a valuable contribution. I am concerned that the original version was lacking in clarity, and glossed over the essential insight of how shaping modifies dynamics. While the rebuttal convinced me that there are positive results there, this could be quite a large change from the version I reviewed.", "rebuttal": "Thank the reviewer for the detailed and valuable feedback very much. Our point-by-point replies are below.\n\n***On Weaknesses:***\n\n1. In our study, shaping plays a critical role in two key ways:  \n(1) It accelerates learning, as shown in Fig. 4c, 5b, and S4.  \n(2) It enables the emergence of abstract solutions that are difficult to reach via direct training. Specifically, in networks without shaping, we observe that odors at P4 and P5 in S2a/S2b often collapse onto shared attractor trajectories at the beginning. Successfully solving the task requires these trajectories to split - a process that is unstable and often fails. In contrast, shaping progressively builds primitive attractors and leverages them to scaffold distinct, structured sequence attractors from the outset, leading to more stable and generalizable solutions.  \n\nExtending this mechanism to other architectures and tasks, and probing its necessity more broadly, are important future directions. We will highlight this in the revised manuscript as a key open question.\n\n2. We agree that the 3-stage shaping protocol is not the only way to achieve representational similarity between RNNs and neural data. In reply of Question 3 (see below), we explored variations of the three-stage design. These alternatives also improve learning efficiency, as primitive attractors reliably emerged and scaffolded subsequent task acquisition. Our work is not to claim exclusivity, but to demonstrate that this protocol offers an effective and biologically inspired framework for learning structured schema. We view prior works [1, 2] as complementary: multi-task training could enrich the repertoire of primitive attractors, which shaping could then organize into higher-level schema.\nWe will revise the manuscript to clarify these.\n\n3. Thank you for the comment. Both tasks share a structural similarity—building complex sequences from simpler components (e.g., phonemes to words, or odor-rewards to odor sequences). Yet, the keyword spotting task incorporates critical real-world variations (e.g., speaker variability in acoustics, timing, pronunciation) absent in the initial task, significantly increasing the generalization challenge. Despite this, the model learn structured schema efficiently, demonstrating its robustness and transferability.\nWe'll clarify it in the revised manuscript.\n \n4. We will clarify training settings and justify key design choices clearly in the revised text.\n\n\n***On Questions:***\n\n1. Thanks for pointing this out. We will clarify the special role of odors 13–14 in the revised task description.\n\n2. We acknowledge that Zhou et al. did not formally report training challenges in the paper. However, the necessity of shaping is a well-known principle among neuroscientists working on rodent behavior and electrophysiology, even for simple tasks (e.g., Fig. 1d in [3]). To improve rigor, we will revise the sentence to: “Zhou et al. adopted a shaping paradigm to train rats to learn the task.”\n\n3. Thank you for the insightful suggestion. We have conducted additional control experiments, summarized in the table below:\n|Training Paradigm     |\tTrials to criterion (Problem 1) \n| :---: | :---:\n| Our shaping (L1 - L2)    |\t  964.6 $\\pm$ 110\n| Alternating (L1-L2-L1-L2) |\t  857.6 $\\pm$ 21\n|Reverse (L2 – L1)        |   1463 $\\pm$ 54\n| No shaping | \t   1725.5 $\\pm$ 75\n\nBoth the alternating and reverse protocols improve learning over no shaping. Notably, alternating can outperform standard shaping, likely by better refining primitive and sequence-level attractors.\n\n4. Thank you for the insightful question.  In our work, as attractor dynamics play a central role in shaping,  we performed control experiments by selectively freezing the recurrent weights ($W^{rec}$) during different training stages and evaluated learning efficiency on Problem 1:  \n(1) Only freezing $W^{rec}$ in L1: The network failed to learn the task, showing that primitive attractor formation is essential at this stage.  \n(2) Only freezing $W^{rec}$ in L2: The network still benefited from shaping (1099 ± 107 trials vs. 1725.5 ± 75 without shaping), but was less efficient than the full 3-stage model (964.6 ± 110), highlighting the role of sequence linking.  \n(3) Only freezing $W^{rec}$ and $W^{out}$ in late phase of L3: The model showed high efficiency for Problem 5 (see Fig. S4), suggesting that once a sequence schema is formed, solving new problems only requires re-binding inputs via feedforward connections—no further recurrent plasticity is needed.  \nThese results align with our attractor shaping framework\n\n5.  (1) Reward prediction accuracy is 99.9 ± 0.02 with shaping vs. 93.2 ± 2.4 without.  \n(2) The loss curves also show slower convergence without shaping.   \n|Epochs | 10 | 30 | 60 | 90 | 120   \n| :--- | :--- | :--- | :--- | :--- | :---   \n| Loss without shaping| 0.32±0.02 | 0.21±0.008 | 0.18±0.006 | 0.17±0.004 | 0.16±0.004   \n| loss with shaping | 0.31±0.06 | 0.1±0.03 | 0.04±0.01 | 0.02±0.006 | 0.015±0.003 \n\n(3) We will include the control model’s RDM and dendrogram in an updated Figure S2.\n\n6. Thank you for the insightful comment. To assess the role of P4–P5 reversal, we conducted two control experiments:\n•\tControl 1: RNNs without shaping, but pretrained on P4–P5 reversal.\n•\tControl 2: RNNs with shaping, but without P4–P5 reversal in L1.\n| Model | Accuracy | RDM Correlation\n| :--- | :--- | :--- \n| without shaping | 93.20±2.4 | 0.861±0.013\n| control 1 | 94.75±4.3 | 0.870±0.008 \n| Shaping  | 99.89±0.03| 0.902±0.021\n| control 2 | 99.30±1.1 | 0.887±0.038\n\nResults show that P4–P5 reversal contributes to improved similarity and accuracy (Control 2 vs. ours), but alone it is insufficient (Control 1 vs. ours). This highlights that broader primitive attractor learning—not just reversal—is crucial.\nIn our framework, P4–P5 reversal supports the formation of distinct attractors for S2a/S2b at P4 and P5, enabling sequence schema construction. Without shaping, the network struggles to resolve overlapping trajectories, as discussed in rely for weakness 1.\n   \n7. Thank you for the question. We vary the input and output values, and the network performs robustly across a wide range parameters, shown in the below table. \n\n(1) Fixed desired output as 2, varying non-zero input value:\n| Input | Accuracy | RDMs Correlation |   \n|  :---  |  :---  |  :---  | \n| 15 | 99.39±0.037 | 0.894±0.018 |   \n| 20 | 99.89±0.034 | 0.902±0.021 |  \n| 25 | 99.97±0.043 | 0.897±0.004 |\n\n(2) Fixed non-zero input as 20, varying desired output:\n| Target | Accuracy | RDMs Correlation |\n| :--- | :--- | :--- |\n| 1 | 99.23±0.098 | 0.874±0.013 |\n| 2 | 99.89±0.034 | 0.902±0.021 |\n| 3 | 99.08±2.366 | 0.896±0.007 |\n\n8. Thank you for the question. Shaping remains effective with higher learning rates (e.g., 99.01 ± 2.7% accuracy, RDM = 0.90 at $5\\times10^{-4}$), but larger rates can disrupt existing attractors. A smaller rate in the schema stage helps preserve stability—consistent with how the brain may regulate plasticity and stability via neuromodulation. We'll clarify this in the revision. \n\n9 and 10. We conducted additional control experiments following Zhou’s shaping protocols:\n•\tControl-9 used the same S1–S2 alternation in our model, and\n•\tControl-10 trained on S1, then 13–14 reversal, then the rest of S2 sequentially.\n\n| Model | Accuracy | RDM Correlation |\n| :--- | :--- | :--- |\n| Our | 99.9±0.03 | 0.90±0.02 |\n| Control-9  | 99.9±0.07 | 0.91±0.017 |\n| Control-10 | 98.3±2.6 | 0.87±0.04 |  \n\nWe found: (1) Control-9 performs on par with our model, suggesting that the specific alternation pattern between S1 and S2 has small effect on schema formation.\n(2) Control-10 performs worse in both reward accuracy and representational similarity, likely due to catastrophic forgetting—when S1 and S2 are trained sequentially, the network struggles to form stable primitive attractors for both. In contrast, biological brains are equipped with continuous learning mechanisms to mitigate such interference.\n\n11. Thank you for the thoughtful question. Rats were not explicitly trained to report odor identity, only to make binary choices. However, we included identity prediction in L1 based on neurophysiological evidence.  \n(1) Our RNN models only the OFC, which may receive supervised signals from other cortical regions.  \n(2) Experimental studies show that OFC neurons exhibit odor selectivity when odor identity is behaviorally relevant, and diminishes when it is not [4].\n(3) In L1, odor identity aids reward prediction, so it's trained. In L2 and L3, reward depends more on sequence context, not identity - so the model no longer needs to predict it.\n\n12. Thanks for pointing this out. The fit is based on simulation data, not experimental results. We will revise the label for clarity and correct the caption error from \"polynomial\" to \"exponential\" in the revised text.\n\n13. Zhou et al. use poke latency as an indirect behavioral measure of learning-to-learn (see their section “Accelerating learning and neural changes”). Additionally, as noted in their abstract, the idea that schema serves as prior knowledge to enhance learning efficiency is treated as a well recognized outcome, supported by prior studies [5].\n \n14.  Thank you for pointing this out. It is a mistake. We will revise the sentence to: “Each major branch is further divided into two sub-branches according to spatial position.”\n\n15. In Fig. S4, our model performs well on Problem 5 even under a larger learning rate. Using a smaller learning rate requires more trials for the shaping network to meet the same criterion, making the comparison more stringent.\n\n16 and 17. Thanks for pointing this out, we will correct the typo (\"corss\" → \"cross\")  and move Figure S7 to the main text in the revised text.\n\n\nReferences: \n\n[1] Driscoll et al., Nature Neuroscience, 2024  \n[2] Turner et al., NeurIPS, 2023  \n[3] Hiroshi Makino, Nature Neuroscience, 2022  \n[4] Robert C. Wilson et al., Neuron 2013     \n[5] Dorothy Tse et al., Science, 2007"}, {"review_id": "MCrZYPiDIV", "reviewer": "Reviewer_GZ49", "summary": "This paper investigates how schema representations can emerge in recurrent neural networks (RNNs) through a shaping-based training protocol. Inspired by experimental paradigms from neuroscience, the authors use a three-stage curriculum (task primitive learning, task sequence learning, and task schema learning) to train RNNs on an odor-based sequential decision task originally developed for rodents. Through this process, the RNN develops low-dimensional sequence attractors that evolve over the shaping phases: from discrete odor-reward associations, to temporally linked sequence attractors, and ultimately to abstract attractor manifolds that generalize across different odor sets. These dynamics replicate key features of schema learning observed in the orbitofrontal cortex, including learning-to-learn behavior, structured representational geometry, and dimensionality compression. The authors further demonstrate that the same attractor-based shaping method improves sample efficiency in a real-world keyword spotting task, suggesting broader applicability beyond the original biological domain.", "questions": "The paper shows that shaping yields structured schema dynamics, but it is unclear whether these attractors emerge consistently across random initializations. Have the authors analyzed whether different networks trained under the same shaping protocol converge to similar internal solutions (e.g., attractor geometry, transition structure), or whether they are one of many functionally equivalent but structurally distinct implementations? Quantifying this could clarify whether the observed schema dynamics are necessary or incidental. Prior work (e.g., Huang et al., 2024; Kepple et al., 2022) highlights this concern.\n\n\nThe framing of \"sequence attractor schema\" is compelling, but conceptually overlaps with earlier accounts of schema learning and compositional representations in RNNs (e.g., Yang & Rajan, 2019; Driscoll et al., 2024). Could the authors clarify how their formulation differs from those prior works — for instance, in terms of mechanisms, representations, or generality — and what is uniquely demonstrated here?\n\n\nThe shaping procedure appears to follow a specific three-phase design. How sensitive are the results to the structure or granularity of this curriculum? For example, would adding more intermediate steps or using a less biologically motivated sequence of subtasks still produce schema attractors? Clarifying this could help assess whether shaping serves primarily as a biological model or as a general training principle.\n\n\nWhat is the role of attractor geometry in generalization? The shaped RNN exhibits low-dimensional attractor dynamics and faster learning on novel sequence tasks. Do the authors have evidence that the geometry of these attractors (e.g., manifolds, fixed points, transitions) directly contributes to generalization, beyond faster convergence? For instance, are new sequences dynamically composed from existing substructures? A deeper analysis here could clarify the functional role of the schema attractor beyond representational structure.\n\n\nWhat is the relevance to broader RNN architectures and tasks? The results are demonstrated in a vanilla RNN trained on a neuroscience-inspired sequence task and briefly extended to keyword spotting. Could the authors comment on whether they expect shaping to induce similar schema dynamics in larger or more structured RNNs (e.g., GRUs, LSTMs, gated architectures, or multi-region RNNs as in the recent neuroscience literature), or in non-sequential domains? Understanding these boundaries would clarify the scope of the paper’s claims and implications.", "limitations": "Yes, though several important contextual gaps remain. The submission presents strong evidence that shaping induces structured, schema-like dynamics in RNNs, but does not address whether these dynamics are consistent across random seeds or training runs. Prior work (e.g., Huang et al., 2024; Kepple et al., 2022) has shown that multiple recurrent network solutions can yield the same behavior while differing internally, raising questions about identifiability and robustness. A discussion of this variability, even in the absence of new experiments, would help calibrate how general the observed attractor structure is.\nIn addition, while the concept of curriculum shaping is well-motivated, its relation to earlier theoretical and modeling work on schema learning, compositional dynamics, and attractor reuse is not fully articulated. Greater clarity on what is conceptually new would strengthen the paper’s broader positioning. Finally, the extension to domains beyond the primary odor-sequence task is limited, leaving open whether the shaping protocol generalizes across architectures or learning settings. These are not fatal flaws, but should be acknowledged in any discussion of the work’s scope.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality: This presents a well-executed series of experiments that investigate how shaping—implemented as a staged curriculum—sculpts internal dynamics in RNNs trained on a biologically inspired sequential decision task. The authors show that shaping leads to attractor structures at multiple scales: discrete odor-specific fixed points, low-dimensional sequence manifolds, and eventually generalized “schema” dynamics. These results are supported by fixed-point analyses, PCA visualizations, and comparisons to orbitofrontal cortex recordings. The use of a neuroscience-motivated shaping protocol (mirroring rat studies) and the emphasis on mechanistic explanation through dynamical systems tools are strengths.\n\nHowever, while the findings are compelling, the paper does not address whether the learned attractor solutions are unique or degenerate across initializations—a key concern given prior work (e.g., Kepple et al., 2022; Huang et al., 2024) showing that multiple RNNs can exhibit equivalent task behavior but diverge in internal structure. Although the authors mention seed averaging, they do not quantify solution diversity or identifiability, leaving open whether the schema attractor is a convergent or incidental outcome of the shaping process. Acknowledging these issues would improve the paper’s rigor.\n\nClarity: The paper is clearly written and well-structured. The three-phase shaping procedure is motivated, defined, and visually summarized. Core concepts—such as “sequence attractors” and “schema manifolds”—are introduced in accessible terms, and the experimental design is intuitive. The dynamical analyses (e.g., attractor structure, dimensionality compression, RNN trajectory geometry) are presented coherently, even for readers outside of neuroscience. Still, the framing of novelty could be made more explicit. Key terms such as “sequence attractor schema” are introduced without a formal definition, and the distinction from earlier work on compositional dynamics, schema learning, or task manifolds (e.g., Yang & Rajan 2019; Kepple et al. 2022) is somewhat implicit. Greater effort to clarify what is conceptually new—especially relative to existing schema modeling frameworks—would help the paper stand on its own.\n\nSignificance: The submitted work addresses a long-standing question in both neuroscience and machine learning: how structured representations and learning-to-learn capabilities can emerge in sequential environments. The use of curriculum (shaping) to elicit reusable internal dynamics, and the demonstration that these dynamics resemble neural phenomena in OFC, make the paper significant for biologically grounded modeling. The generalization of the approach to a keyword spotting task also suggests relevance to applied RNN training. That said, some aspects of the contribution are only confirmatory. The idea that curriculum learning induces reusability and compositionality is well established (in both biological and artificial systems), and prior work has shown that networks can learn task-aligned manifolds and latent dynamics under suitable conditions. Without quantifying generalization across domains or linking the observed schema attractors to improved task-level abstraction or zero-shot performance, the submission’s broader implications remain suggestive but incomplete.\n\nOriginality: This submission introduces a new synthesis of ideas. It applies a structured shaping protocol to sculpt attractor-based schema representations in RNNs and connects this process to known neurophysiological dynamics. The concept of progressively building attractors—from primitives to sequences to abstract schemas—via curriculum is well-illustrated and not previously demonstrated in this form. However, several foundational ideas in the paper have appeared in earlier work, including compositionality (Yang & Rajan, 2019), degeneracy in internal solutions (Huang et al., 2024), and shaping effects on internal structure and generalization (Kepple et al., 2022). While the current work builds productively on these foundations, the submission does not fully acknowledge this overlap. A clearer positioning would help delineate what is newly shown (e.g., sequence-level attractors shaped by curriculum) versus what conceptually recurs under different names.", "was_revised": true, "final_justification": "The rebuttal addressed several technical points, including convergence across seeds, architectural robustness, and shaping schedule variants. These additions improve the paper, but they do not fully resolve the broader concerns about novelty, generality, and positioning relative to prior schema/compositional work. Overall, the paper is technically solid and conceptually interesting, but I remain at a borderline accept given the remaining limitations.", "rebuttal": "Thank the reviewer for insightful review! Your suggestions were very helpful and significantly improved our work. We'll address each of your concerns point by point.\n\n***On Weaknesses and Questions：***\n\n**1. Convergence Across Random Seeds（for quality concern and question 1）**\n\nThank you for the insightful comments. Our analyses suggest that shaped RNNs exhibit convergent internal dynamics regardless of initialization. As shown in Fig. S6, we used Canonical Correlation Analysis (CCA) to compare RNNs trained with different seeds. Fig. S6a demonstrates strong alignment of the top three canonical components across models, indicating consistent attractor representations. Moreover, these components exhibit interpretable structure consistent with neural data (Fig. S6a,b). We also visualized the attractor trajectories across seeds and found qualitatively similar sequential dynamics (similar to Fig. 3d), supporting the emergence of a shared schema manifold. We will include these additional visualizations in the revised version.\n\nAdditionally, our findings are not in conflict with prior work [1,2,3], which shows that solution degeneracy can be mitigated by task complexity, input-output richness, or multi-task learning. Our task, by design, involves multi-problem learning and a shaping process that may naturally reduce degeneracy.\n\nOverall, we acknowledge that our current analysis spans a limited set of RNN configurations. In the revision, we will explicitly discuss this limitation, cite relevant work [1, 2, 3], and highlight the need for broader analyses to assess identifiability across architectures and task settings.\n\n**2. Distinction from Prior Schema and Compositionality Work (for clarity and question 2)**\n\nThank you for the helpful suggestion. We will clarify the definition of “sequence attractor schema” in the revised manuscript. Specifically, it refers to a structured dynamical motif: a progression of discrete attractor states connected via low-dimensional manifolds, forming a reusable scaffold for representing abstract sequential structure.\n\nOur formulation differs from prior schema-learning frameworks (e.g., Yang & Rajan, 2019; Driscoll et al., 2024) in three main aspects:  \n(1) Mechanism: Previous studies primarily induce schemas through multi-task learning, extracting common motifs across tasks. In contrast, our schema emerges through curriculum-based shaping, where complex sequential dynamics are gradually built from simpler attractor primitives in a stage-wise manner.  \n(2) Representation: Driscoll et al. describe various dynamic motifs, including discrete and continuous attractors or decision boundaries. Our schema is a sequential attractor—a trajectory-like structure composed of discrete fixed points and the low-dimensional flow between them. It captures not just static representations or decisions, but structured temporal dynamics.  \n(3) Generality: Prior schemas are typically applied to memory, binary decision-making, or integration tasks. In contrast, our sequence attractor schema supports abstract sequential composition, offering potential applications in domains like hierarchical motor generation [5] or syntactic structure in language [6]. This could significantly expand its utility beyond traditional schema functions.  \nOverall, we view our work as complementary to earlier efforts. For instance, multi-task learning (as in Driscoll et al.) could enrich the set of primitive motifs used in shaping. Conversely, when combined with our sequence attractor schema, it could enable the formation of more diverse and complex schema attractors for intricate cognitive tasks, such as rule-based sequential decision-making.  \nWe will clarify these distinctions and conceptual contributions more explicitly in the revised text.\n\n**3. Three Phase Design Robustness (for question 3)**\n\nThank you for the insightful suggestion.  To evaluate the sensitivity of results to the three-phase shaping design, we conducted additional control experiments. Specifically, we reversed the order of stage 1 (L1) and stage 2 (L2), or alternated L1 and L2 multiple times. The results are summarized in the table below.  \n\n| Paradigms |   Trials to criterion for P1 (mean)    |  RDMs' correlation    \n|  :--: | :--: | :--:   \n| No shaping     | 1725.5 $\\pm$ 75  | 0.86 $\\pm$ 0.02 \n| Alternating (L1-L2-L1-L2)\t |    857.6 $\\pm$ 21  | 0.91$\\pm$ 0.02\n| Reversing (L2-L1)\t         |  1463 $\\pm$ 54  |   0.89 $\\pm$ 0.01\n| Our shaping (L1 - L2)\t    |  964.6 $\\pm$ 110   |   0.902$\\pm$ 0.02\n\nAll variants outperform direct training, confirming that shaping facilitates the formation of primitive attractors that scaffold subsequent learning. Notably, the alternating schedule yields the best performance, likely because it more effectively reinforces and integrates both primitive and sequence-level attractors during optimization.\n\nThis flexibility also suggest that our proposed shaping-induced dynamic framework is not tied to a rigid sequence of stages. For instance, the acquisition of diverse primitive attractors could also be achieved through multi-task learning [4]. Moreover, such a shaping idea has recently been applied to motor sequence training in monkeys [5], underscoring its broader applicability.\n\nWe will include these new control results in the supplementary material and expand the discussion in the revised manuscript.\n\n**4. Functional Role of Attractor Geometry in Generalization (for significance and question 4)**\n \nThank you for the insightful suggestion. We agree that clarifying the functional role of attractor geometry in generalization is an important direction.\nTo investigate this, we conducted experiments where the recurrent and readout weights were frozen, and only the feedforward input weights were trained on a new odor-sequence task (see Fig. S4). The network rapidly adapted to the new task with significantly fewer trials.  We further visualized the RNN dynamics and confirming that new stimuli are flexibly mapped onto pre-existing attractors, effectively reusing learned substructures. These results suggest that the existing attractor geometry supports fast generalization. \n\nBeyond our specific task, the shaping-induced attractor geometry may generalize to other sequence domains, such as motor generation [5] and the syntactic structure of language [6]. For example, recent work [5] showed monkeys trained via a similar shaping process could learn primitive strokes leading to complex character drawing. This complex motor generation could potentially be explained by our sequence attractor schema framework.\n\nWe will add these new results and expand the discussion of attractor geometry and its role in generalization in the revised manuscript.\n\n**5. Architectural Generality of Schema Attractors (for question 5)**\n\nThank you for the thoughtful suggestion. We conducted additional experiments using GRUs and LSTMs on the same odor-sequence task. As summarized in the table below, \n\n| Network\t | RDMs Correlation \n|  :--: | :--:  \n|LSTM with shaping\t | 0.89 ± 0.01  \n|LSTM without shaping\t | 0.87 ± 0.03  \n|GRU with shaping\t | 0.906 ± 0.01  \n|GRU without shaping\t | 0.87 ± 0.007\n|RNN with shaping\t | 0.902 ± 0.02  \n\nOur results suggest that shaping accelerates learning (shown in Fig.4c) and produces RDMs that closely align with neural data. Furthermore, our visualizations of attractor dynamics demonstrate that GRUs and LSTMs can achieve attractor dynamics comparable to those seen in vanilla RNNs. These findings indicate that the sequence attractor schema induced by shaping generalizes across different recurrent architectures, including gated models.\n\nWe will include these additional results in the supplementary material and clearly outline the extension of our work to multi-region RNNs or non-sequential domains as future work in the revised manuscript.\n\n***On Limitations:***\n\nWe appreciate the thoughtful summary and agree that clarifying the scope and limitations of our findings is essential. We've addressed each key point below:\n\n(1) To assess the consistency of schema dynamics across initializations, we performed Canonical Correlation Analysis (CCA) and visualized attractor trajectories for networks trained with different seeds. As Figure S6 shows, models exhibit highly aligned low-dimensional attractor geometries, indicating convergence toward a shared internal solution under shaping. We acknowledge that our analysis is currently limited to vanilla RNNs, and a broader evaluation across architectures remains a future direction.\n\n(2) Our sequence attractor schema distinguishes itself from previous schema-learning frameworks in several ways: its mechanism (shaping versus multi-task/meta-learning), its representation (sequence attractors versus discrete or continuous attractors), and its generality (capturing compositional sequence structure applicable across various sequence task domains). We view this as a complementary advance and will clarify these conceptual differences in the revision.\n\n(3) Our additional results demonstrate that shaping induces similar attractor dynamics and structured representation geometry in both GRUs and LSTMs. While these results support architectural generality, we have not yet tested multi-region models or non-sequential tasks. We will explicitly define this boundary in the revision and include new results in the supplement.\n\nWe will expand the discussio to clearly acknowledge these limitations, cite relevant prior work (e.g., [1,2,3]), and highlight directions for generalizing the framework.\n\nReferences:  \n[1] Kepple et al., NeurIPS, 2022  \n[2] Huang et al., arxive, 2025  \n[3] Yang & Rajan, 2019  \n[4] Driscoll et al., Nature Neuroscience, 2024  \n[5] Lucas Y. Tian et al., bioRxiv, 2025  \n[6] Stanislas Dehaene et al., Neuron, 2015"}, {"review_id": "PtmcnrX8gC", "reviewer": "Reviewer_nFhk", "summary": "The authors present a method for training recurrent neural networks (RNNs) on an odor-reward association task using a ‘shaping procedure’. This procedure, inspired by how rats are trained on the same task, is implemented in three stages. The authors show that shaping leads to faster learning compared to training without shaping, promotes the development of structured task representations, and results in a progressive reduction in the dimensionality of network activity over training. They further analyze the network dynamics, showing a transition from isolated discrete attractors to sequential attractors, and eventually to more abstract sequence attractors. This progression is proposed as a novel perspective on schema formation in biological neural systems. The approach is also applied to a keyword spotting task, where similar patterns emerge, supporting the generality of the findings.", "questions": "(1) Figure 2(e) shows that shaping improves the correlation between the representational dissimilarity matrices (RDMs) of the model and the data. However, even without shaping, the model still achieves a relatively high correlation and also, a high correlation does not immediately mean that it is a well structured representation. Could you explore or provide another way to demonstrate that shaping is essential for developing a well-structured representation? For example, is there a qualitative difference in the types of attractors or representations formed with vs. without shaping?\n\n(2) The paper shows that shaping leads to a reduction in the dimensionality of the network's activity. Could you also show whether a similar reduction does not occurs in models trained without shaping? This would clarify whether the reduction in dimensionality is specifically a result of shaping or a more general consequence of learning.\n\n(3) In the first stage of training, the RNN is trained to output the odor identity in addition to reward/non-reward. This seems to differ from the animal experiment or were rats also required to classify all 16 odors, or was the task limited to a binary choice (rewarded vs.  non-rewarded)? If the animals were not trained to output odor identity, please explain more clearly why this added output was included in the model and why the comparison to experimental behavior remains valid. Also, would the shaping procedure still be effective if the model were only required to predict reward/non-reward, as in the behavioral task?\n\n(4) You state that shaping leads to more efficient or faster learning (learning-to-learn, Fig., 2(b)). Did you control for how many entirely new odors are introduced in each subsequent problem? If the number of novel odors decreases over time, this could itself contribute to faster learning as the task effectively get easier (less new odors) rather then the network better at learning. Please consider adding a control to distinguish these effects.\n\n(5) L215ff: In this section, you analyze attractor convergence by turning off all inputs. However, since the attractors depend on input, this approach may not reflect where the attractors are under task-relevant conditions. Please clarify this analysis and how it relates to behaviorally meaningful network states. Also, does this approach imply that you are training or tuning the initial conditions of the network during training? If so, how does this relate to the experimental setup with rats?", "limitations": "(see questions above)", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The structuring of shaping as a learning path is interesting and well thought out. It's particularly nice that the approach is grounded in experimental data and aims to closely follow and be compared to the animal training protocol. However, in its current state it provides only limited evidence that shaping is crucial to achieve the observed properties such as abstract structured representations but mainly shows that also with shaping these properties emerge.\n\nThe concept of shaping described in the paper is closely related to curriculum learning in machine learning, where tasks are broken down and gradually made more difficult over training to facilitate and guide learning. This connection is currently not very clearly referenced or discussed. \n\nThe paper is well structured and some minor clarifications would help to avoid some misunderstandings:\n* Lines 84 and following: It would be helpful to state more clearly that the initially learned values associated with different odors do not persist throughout the sequences. This point only becomes fully clear later in the text or by closely analyzing the figure, which cause a lot of confusion for me.\n* Please clarify what specific behavior was expected from the rat at each training stage. A more explicit description would make it easier to compare the experimental protocol with your model and better understand how the shaping stages align.\n* Line 124: You mention that Gaussian noise was added, but it would improve clarity to include this term directly in Equation 1 (and/or Equation 2), so the reader can see clearly how noise is incorporated into the model dynamics.", "was_revised": true, "final_justification": "The authors provide a compelling computational account of shaping and schema formation, grounded in dynamical systems and aligned with biological data. Their rebuttal clarified conceptual and methodological points, added useful analyses, and strengthened links to existing work. The additional experiments address concerns around the necessity and impact of shaping, supporting the main claims. While some limitations remain, this is a valuable contribution.", "rebuttal": "We thank the reviewer for valuable comments .We'll address each of your concerns point by point.\n\n***On Weaknesses:***\n\n1. Thank you for the valuable feedback. Shaping plays a critical role in two key ways:  \n\n(1) It accelerates learning, as shown in Fig. 4c, 5b, and S4.  \n(2) It enables the emergence of abstract structured representations robustly that are difficult to reach via direct training, as detailed in reply in question 1 (see below). \n\nSpecifically, in networks without shaping, we observe that odors at P4 and P5 in S2a and S2b often collapse onto shared attractor trajectories and leads the network to stack at a local minima. In contrast, shaping progressively builds primitive attractors and leverages them to scaffold sequence structure learning, leading to more stable and generalizable solutions.  \n\nWe will clarify this in the revised manuscript and highlight it as a key open question.\n\nMore broadly, our shaping—from primitives to sequences to schemas—is widely used in animal training (e.g., in recent monkey handwriting-drawing generation studies [1]), yet its dynamical underpinnings remain poorly understood; this \"elephant in the room\" has lacked a concrete computational model. Our work provides, to our knowledge, the first mechanistic account of shaping-induced schema formation grounded in attractor dynamics and directly comparable to neural data. \n\n2. Thank you for this insightful comment. We'll clarify these connections in the revised manuscript as follows:\n\nOur shaping approach shares broad similarities with curriculum learning in its factorization of complex tasks into subtasks to improve learning efficiency [2,3]. However, curriculum learning often focuses on performance generalization through methods like task factorization [3], sample ordering [3], or automated curriculum generation [3,4]. These approaches typically lack explicit consideration of how internal representations are structured or how underlying abstract knowledge is learned [3].In contrast, our shaping framework is grounded in dynamical systems theory and biological learning paradigms. Our primary aim is to guide the emergence of structured attractor dynamics—progressing from primitive attractor structures to abstract attractor schemas. These schemas serve as reusable abstract knowledge that can scaffold learning and enable generalization across diverse task domains [5,6]. This mechanistic perspective may offer potential inspiration for new curriculum learning algorithm that explicitly facilitate the learning of structured, generalizable knowledge.\nWe will incorporate this discussion into the revised text.\n\n3. Thank you for the valuable suggestions. We will make the following clarifications in the revised text:  \n(1) explicitly state that the initially learned values associated with different odors do not persist throughout the sequences.  \n(2) add an explicit summary of the expected rat behavior across the three shaping stages, aligned with the model design.   \n(3) explicitly include the Gaussian noise term in Equation 1 to clearly show how stochasticity is incorporated into the model dynamics.\n\n***On Questions:***\n\n1. Thank you for the insightful comment. To address the concern, we performed additional quantitative and qualitative analyses, which show that shaping is essential for forming structured and generalizable attractor representations.\n\n**Behavioral divergence:**  Although both shaped and unshaped RNNs have high RDM correlation with neural data, their behavioral performance is significantly different. The unshaped networks often perform suboptimally. For example, reward prediction accuracy is much higher with shaping (99.89 ± 0.03%) than without (93.20 ± 2.4%). This difference is most apparent when predicting odor rewards at P4 and P5 in S2a/S2b, as reward contingencies are reversed. Here, the shaped network performs reliably (98.6 ± 4.61%), while the unshaped network performs close to chance (56.8 ± 31.01%), indicating it failed to handle this critical reversal.\n\n**Attractor dynamics differ qualitatively:** Visualizing attractor trajectories further clarifies the difference. In shaped RNNs, odors at P4 and P5 in S2a / S2b are already represented by four distinct attractors during primitive learning, which are then gradually linked to form sequence-specific attractor trajectories.\n\nIn contrast, RNNs without shaping lack this scaffold: odors at P4 and P5 across S2a and S2b are initially represented using a shared attractor trajectory. To succeed on the task, the network must later split this trajectory—a process that is often unstable and results in suboptimal convergence.\n\nThese results demonstrate that shaping not only improves representational similarity but also enables the formation of well-separated, sequence-specific attractor manifolds critical for solving the task. We will include these new analyses and visualizations in the revised manuscript.\n\n2. Thank you for the helpful suggestion. We conducted the same dimensionality analysis on RNNs without shaping and found that although dimensionality reduction also occurs, it differs fundamentally from that with shaping condition.\n\n| Epochs  |  0  |  10  |  20  |  30  |  40   \n| :---: |   :----:  | :---: | :--: | :---: |  :--:\n| P1 | 0.30 $\\pm$ 0.01 |  0.94 $\\pm$ 0.003 | 0.93 $\\pm$ 0.002 |  0.93 $\\pm$ 0.003 | 0.93 $\\pm$ 0.002\n| P2 | 0.90 $\\pm$ 0.04 |  0.92 $\\pm$ 0.005 | 0.93 $\\pm$ 0.006 | 0.93 $\\pm$ 0.004  |0.93 $\\pm$ 0.003\n| P3 |   0.77 $\\pm$ 0.037 | 0.92 $\\pm$ 0.005 | 0.94 $\\pm$ 0.003  |0.94 $\\pm$ 0.003 | 0.94 $\\pm$ 0.002\n\nFirst, RNNs without shaping reach a low-dimensional regime much earlier(93% without shaping versus 82.5% with at P1) - even in P1—suggesting premature compression. However, this reduction does not reflect meaningful abstraction, but rather an early convergence to coarse, shared representations that fail to distinguish sequence-specific differences (e.g., odors 13–14 in S2a/S2b).\n\nSecond, unlike shaped RNNs, which exhibit progressive dimensionality reduction across tasks (P1 to P3) consistent with schema abstraction [7] (averaging problem-specific features and preserving shared structure), RNNs without shaping show little further change, indicating a lack of representational refinement.\n\nOverall, with shaping, compression reflects a structured abstraction built on pre-established attractors. Without shaping, however, compression is shallow and leads to suboptimal generalization. We will include these results and clarifications in the revised manuscript.\n\n3.  Thank you for the thoughtful question. In the animal experiments, rats were not explicitly trained to report odor identity—only to make a binary decision. However, we included odor identity prediction in Stage 1 of the model based on the neurophysiological evidence.  \n(1) Our RNN models only the OFC, which may receive supervised signals from other cortical regions.  \n(2) Experimental studies have shown that OFC neurons exhibit odor selectivity when odor identity is behaviorally relevant, and this selectivity diminishes when it is not [8].   \n(3) Consistent with this, in Stage 1 of our model, odor identity is informative for reward prediction and thus included as a target. In contrast, in Stages 2 and 3, reward depends on sequence context rather than odor identity (rewards at P4 and P5 in S2a/S2b can not be predicted by odor identity ), so identity prediction is no longer required.\n\nAdditionally, we also conducted a control experiment where the model in Stage 1 was trained only to predict reward, without odor identity (Control model), shown in below.\n\n|Model  |           Trials to criterion (P1) \n| :--: |:--:\n|Without shaping |   1725.5 $\\pm$ 75\n|Control     |  1259.33 $\\pm$ 130.3         \n|Ours          |  964.6 $\\pm$ 110   \n\nWhile the control model improves learning efficiency over the no-shaping baseline, it is still less effective than our full model that includes odor identity prediction. This suggests that explicitly training odor identity helps the network develop richer primitive attractors, which in turn facilitate sequence learning in later stages.\nWe will clarify this rationale and include the control results in the revised manuscript.\n\n4. In each new problem, 16 entirely novel odors are introduced—none of which appeared in the previous problems or during the shaping phase. Therefore, the observed improvement in learning speed cannot be attributed to a decreasing number of novel stimuli. We will clarify this point in the revised text.\n\n5.  Thank you for the thoughtful comment. We clarify the attractor analysis as follows:\n（1）In the task, each odor is presented briefly (as a localized input bump), followed by a delay with no external stimulus—just background noise. Our attractor states capture the internal representations during these delay periods.\n（2）The uncovered attractor states in attractor analysis form a latent scaffold that organizes sequential processing. As shown in Fig. 3d, these states correspond well with the actual neural trajectories during task execution, indicating that they guide task-relevant dynamics. Each attractor represents a specific odor-event memory, and transitions between them reflect the evolution of task state as new odors are presented.\n（3）This approach is consistent with prior work that uncovers low-dimensional attractor structures underlying neural computation [2,3].\n\nWe do not train or tune the initial state of the network. It is fixed to zero across all training and evaluation phases (see line 7, Appendix), aligning with the experimental condition where rats begin trials from a neutral state.\n\n\n\nReferences  \n\n[1] Lucas Y. Tian et al., bioRxiv, 2025  \n[2] Bengio et al.,ICML, 2009   \n[3] Xin Wang et al., PAMI, 2021  \n[4] Jakob Bauer et al.,ICML,2023  \n[5] Mikail Khona, Nature Reviews Neuroscience, 2022  \n[6] Driscoll et al.,Nature Neuroscience,2024  \n[7]  Oded Bein et al., nature reviews neuroscience,2024  \n[8] Robert C. Wilson et al., Neuron 2013"}]

Author rebuttal:

Meta-review: This paper model schema learning via training RNNs with shaping, which is inspired by an odor-sequence task on rats (ref. 4). The schema learning contains three stages that gradually emerge sequence attractors, and the learned hidden representation is similar to the neural data. 

All reviewers think the paper addresses an important question in both neuroscience and machine learning and give scores above acceptance. Combined, I recommend accepting this paper. Please combine reviewers’ feedback and the new experiments done during the rebuttal periods to revise the manuscript. Additionally, adding some insightful discussions in the rebuttal would further improve the paper. For example, reviewers think the paper only focuses on reproducing a specific experiment, while a more fundamental question is how shaping works and why it is necessary. And whether the specific 3-stage training is the unique way to reproduce neural data.
Decision: Accept (poster)