Efficient Allocation of Working Memory Resource for Utility Maximization in Humans and Recurrent Neural Networks
Authors: Qingqing Yang
Hsin-Hung Li
Venue: NeurIPS 2025 poster ()
Keywords: working memory
reward maximization
biologically inspired networks
efficient coding
recurrent neural networks
Primary area: neuroscience_and_cognitive_science
TL;DR: Humans and recurrent neural networks both allocate working memory resource to maximize utility, shaped by natural stimulus statistics and learned stimulus-reward association.

Abstract: Working memory (WM) supports the temporary retention of task-relevant information. It is limited in capacity and inherently noisy. The ability to flexibly allocate WM resource is a hallmark of adaptive behavior. While it is well established that WM resource can be prioritized via selective attention, whether they can be allocated based on reward incentive alone remains under debate—raising open questions about whether humans can efficiently allocate WM resource based on utility. To address this, we conducted behavioral experiments using orientations as stimuli. Participants first learned stimulus–reward associations and then performed a delayed estimate WM task. We found that WM precision, indexed by the variability of memory reports, reflected both natural stimulus priors and utility-based allocation. The effects from reward and prior on memory variability both grew over time, indicating their effects in stabilizing memory representations. In contrast, memory bias was largely unaffected by time or reward. To interpret these findings, we extended efficient coding theory by incorporating time and reformulating the objective from minimizing estimation loss to maximizing expected utility. We showed that the behavioral results were consistent with an observer that efficiently allocates WM resource over time to maximize utility. Lastly, we trained recurrent neural networks (RNNs) to perform the same WM task under a 2×2 design: prior (uniform vs. natural) × reward policy (baseline vs. reward context). Human-like behaviors emerged in RNNs: memory was more stable (lower variability) for stimuli associated with higher probability or rewards, and these effects increased over time. Transfer learning showed that recurrent dynamics were crucial for adapting to different priors and reward policies. Together, these results provide converging behavioral and computational evidence that WM resource allocation is shaped by environmental statistics and rewards, offering insight into how intelligent systems can dynamically optimize memory for utility under resource constraints.

Efficient Allocation of Working Memory Resource for Utility Maximization in Humans and Recurrent Neural Networks
Qingging Yang Department of Psychology The Ohio State University Columbus, OH 43214 yang. 61188osu. edu
Hsin-Hung Li Department of Psychology The Ohio State University Columbus, OH 43214 1i. 14442409uuuu.. edu
Abstract
Working memory (WM) supports the temporary retention of task- informa- tion. It is limited in capacity and inherently noisy. The ability to flexibly allocate WM resource a hallmark of adaptive behavior. While it is well established be prioritized via selective attention, whether they be
that WM resource allocated based on reward incentive alone remains under debate -aiiiing open questions about whether humans can efficiently allocate WM resource based on utility. To address this, we conducted behavioral experiments using orientations as stimuli. Participants first learned stimulus- a delayed estimate WM task. We found that WM precision, indexed by the vari- ability memory reports, reflected both natural stimulus priors and utility-based allocation. The effects from reward and prior on memory variability both grew over time, indicating their effects stabilizing memory representations. memory bias was largely unaffected by time reward. To interpret these findings, we extended efficient coding theory by incorporating time and reformulating the objective from minimizing estimation loss to maximizing expected utility. We showed that the behavioral results were consistent with an observer that efficiently allocates WM resource over time to maximize utility. Lastly, we trained recurrent neural networks (RNNs) to perform the same WM task under a 2x2 design: prior (uniform vs. natural) x reward policy (baseline vs. reward context). Human-like behaviors emerged in RNNs: memory was more stable (lower variability) for stim- uli associated with higher probability rewards, and these effects increased over time. Transfer learning showed that recurrent dynamics were crucial for adapting different priors and reward policies. Together, these results provide converging behavioral and computational evidence that WM resource allocation is shaped by environmental statistics and rewards, offering insight into how intelligent systems can dynamically optimize memory for utility under resource constraints.
associations and then performed
contrast,
Introduction
Working memory (WM) is the neural and cognitive process that temporarily maintain task-relevant information online (addeley 2003 It supports wide range of high-level cognition functions such as learning, decision-making and planning [Collins and Frank 2012 Daneman and Carpenter 1980
SüB al. 2002 Ehrlich and Murray 2022). WM has limited capacity and inherently noisy the
quality of the WM decreases with the number of items stored [Luck and Vogel 1997 2013 Ma et al.
2014. and the precision of WM representations decays time [Perzov al. 2017 Panichello
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

et al., 2019 Shin et al. 2017]. Thereby, the ability to flexibly and efficiently allocate WM resource is essential for adaptive and goal-directed behaviors.
It is well-established that humans can allocate WM resource in response to attentional cues. When multiple items must be remembered, people can prioritize certain items over others, suggesting degree of control over the distribution of WM resource Zhang and Luck, 2008 Emrich et al. 2017
Dube et al. 2017 Klyszejko et al. 2014 Yoo al. 2018 Bays 2014]. However, a more fundamental and still unresolved ved question is whether WM resource is allocated elficiently to maximize reward or
utility- the common currency of the subjective value. Recent studies have reported mixed results regarding whether WM resource can be modulated based on reward alone (Brissenden et al. 2023
Van den Berg et al., 2023 Weiss et al.) [025] Manga et al.) 2020). when factors such as spatial attention are controlled. These findings cast doubt on the notion that WM resource can be flexibly optimized to maximize an individual's utility. parallel, although both cognitive models (Van den Berg et al., 2014] BBas et al. 2024) and recurrent neural network (RNN) models have been developed to account for various aspects of working memory [Wimmer et al., 2014 Compte et al.] [000 Bouchacourt and Buschman 2019 Esnaola-Acebes et al., [022] Wang 2021J. it remains largely
underexplored how such models should allocate WM resource in order to maximize expected utility,
signature of normative and reward-sensitive computation.
Here, we conducted behavioral experiments to test the hypothesis that after value learning, humans allocate WM resource efficiently according to Himuluus-rwart association in order to maximize utility. To preview, we found that memory representations were more stable for stimuli associated with higher reward, as well as for those more probable in the natural environment. To explain these findings, we extended efficient coding theory to the time domain and reformulated the objective from minimizing estimation loss to utility maximization, providing a normative account of how WM resource can be allocated over time to optimize expected utility (Wei and Stocker 2015 Hahn and Wei 2024 Morais and Pillow 2018, Schaffner et al.) 2023]. Finally, we show that recurrent neural networks (RNNs) trained to maximize reward exhibit human-like behaviors, integrating both prior and reward information in manner consistent with the behavioral data.
2 Joint effects of prior and reward on human working memory
2.1 Behavioral tasks
The behavioral experiments consisted of two sessions scheduled on separate days, with one context tested per day (Figure (A). On each day, participants first completed a value learning task followed by a WM delay-estimation task under the same context. The order of the contexts was counterbalanced across subjects (n=14, power analysis see A.1.2)). In the baseline context, all orientations were associated with the same reward; in the reward context, reward increased with the diagonality of the orientations (Figure (BB). We designed the reward context such that the reward structure goes against the environmental distribution of the orientations (Figure [[), where cardinal orientations are more prevalent Girshick et al. 2011] This allowed us to disentangle the impact of the environmental prior and learned reward.
In the value-learning task (Figure I)), each trial started with four gratings with orientations sampled from the environmental distribution presented in different quadrants on the screen. Participants used mouse clicks to choose one grating and received feedback of the rewards (indicated as points). They were instructed to maximize the reward while learning the reward-orientation association. Each participant completed 5 blocks of 20 trials. At the end of the training, participants learned to choose the most diagonal grating in the reward context, while they choose the most diagonal one at a chance level in the baseline context (Figure Dp. details of learning trajectory in A.I.I.
After the value-learning task, participants performed the WM delay-estimation task (Figure [T)). Participants viewed a centrally presented grating (the target, 0.5s duration), and reproduced its orientation after a Is or 5s delay. Orientations of the targets in each run were pseudo-randomly drawn from a uniform distribution. The rewards (points) they earned on each trial decreased linearly with the absolute memory error (the reported orientation minus the true orientation) both contexts. The orientation of the target determined the maximum reward (when memory error = 0²) that can be earned in each trial. Following the same policy they just learned (Figure B in the reward context, more diagonal targets were associated with higher reward, while the rewards were uniform across

---

A
Reward policy
Stimulus distribution
Baseline context (session)
Value-leammng task
WM delay estimation
Reward
Probability
(Environmental dist.)
(Uniform dist.)
Reward context (session)
90
180
BD
180
Value-leaming task
WM delay estimation
Orientation(")
Uniform Orientation(")
(Environmental dist.)
(Uniform dist.)
Baseline context
Reward context
Environmental dist.
D Value-learming task
Learning outcome
E
WM delay estimation
PiChoose the most
diagona aoonnalii Item)
50
0.25
chance
level
stimuli & response
Baseline
Reward
WM target (0.58)
delay (1s or 5a)
60 pts!
feedback (18)
report
feedback (1s)
Figure 1: Behavioral experiments and procedures. (A) The experiments included 2 days, each day assigned to distinct reward policy. (B) Reward policy for the baseline and the reward context. (C) Uniform distribution and environmental distribution of orientations adopted from Girshick et al.
[20TT. (D) Left: The value-learning task: participants learn the reward-orientation association in 4AFC decision task. Right: The learning outcome of the value-learning task. Participants learned to choose the most diagonal orientation the reward context, while the choice is random in the baseline context. (E) The WM delay- estimation task, where the memory of a single orientation is tested after Is 5s delay.
orientations in the baseline context. By the end of each session (day), the total rewards (points) earned across the two tasks were converted into cash bonus earned by the participants.
2.2 Behavioral results
Reward stabilizes WM representations
A
Delay 1s
Delay 5s
B
Variability
Variability (*)
10
Amplitude (*)
two-way interaction
p .05
E
target
Probabilty
1
D
DDlaa 1s Delay 5e
Blas
viriaaillity
Baseline context Reward context
Baseline context Reward context
Magnitude (*)
bias
Bias (*)
Orientation
4
1s
Baseline context
-5
Reward context
135 Orientation (*)
180
Delay 1s Delay 5s
Orientation (?)
Figure 2: Behavioral results. (A) WM variability (bias removed) in each context x delay condition. Data points represent mean + 1 s.e.m. Lines fitted variability (sinusoidal) functions. (B) Vari- ability amplitude fitted from sinusoidal functions. Higher amplitude indicates a stronger oblique effect-cardinall having lower variability compared to the oblique orientations. Error bars represent + 1 s.e.m. (C) WM bias in each context x delay condition. Data points represent mean + s.c.m. Lines are fitted polynomial functions.
(D) Bias magnitude estimated from polynomial function, where higher value denotes higher repulsion away from the cardinals. Error bars represent # s.c.m. (E) Illustrations of how bias and variability are decomposed from memory responses.

---

It is well-established that orientation estimation exhibits distinct patterns of bias and variability (Talor and Bays, 2018 WWii and Stocker 2017 Girshick et al. 2011]. We decomposed the memory emorr into these two components (Figure 2 For each context x delay condition, we fitted polynomial regression the (signed) memory error to predict the bias. After subtracting the predicted bias from the error, we used the standard deviation of the residuals as the variability (details in A.1.5) For visualization, the results were binned based on the orientations of the WM target in Figure
For the variability, we found a significant 3-way interaction indicated by a linear mixed-effects model (context x delay x orientation, p < .05; Figure 2 In the short delay, there was a well-known "oblique effect" [Appelle] 1972] Girshick et al. 2011, Gardelle et al., 2010]: the variability was lower for cardinal compared to the diagonal orientations in both contexts (no interaction of context and orientation,
.16). In the long delay (5s), there was a significant interaction of context and orientation (p < .05), where the variability of the diagonal orientations was lower in the reward context compared to the baseline. That is, after value learning, high-reward (diagonal) orientations were maintained better, reflecting the effect of learned stimulus-rewarr association on stabilizing WM. Interestingly, When the reward is uniform, the oblique effect became more pronounced in the long delay, while the cardinal orientations were more stable (also see Figure 3 where the variability is grouped by contexts). Therefore, more prevalent stimuli in the environment not only were encoded more precisely, but maintained better in WM. In contrast, in the reward context, the oblique effect was attenuated over time, due to the reward structure that opposed the environmental prior. Overall, our results reveal a joint effect of reward and natural stimulus prior on the variability WM.
We further confirmed the above effects by fitting the variability pattern by sinusoidal functions over orientations (details 1.6).. The parameter "oblique amplitude" captured the relative advantage in variability at cardinal compared to the oblique orientations (Figure(33. As result, we found significant 2-way interaction of context and delay (p < .05), and no main effect of context (p = .19) nor delay (p- .20).
For the bias, we observed repulsion bias, with the reported orientations shifted away from the cardinal orientations (Taylor and Bays] 2018 Wei and Stocker 2017 Girshick et al., 2011]. A linear mixed-effect model showed no significant 3- 3-way interaction between context, delay. and orientation (p = .09, Figure 22), and the bias pattern was not affected by either the delay or the reward context. We further quantified the bias magnitude (Figure D details in AA.144). and found no interaction between context and delay (p .78), nor any main effect of context (p .75) or delay (p .95). Given that reward and delay predominantly affected variability, below we focused on modeling their effects memory variability.
WM resource allocation shaped by prior and reward
A
Baseline context
Reward context
B
Baseline context
Reward context
10
15
56
16
Resource (J)
85
1s
Ss
1s
5s
Variability
S
0
45
90
135
180
0
45
90
135
180
45
90
135
180
45
90
135
180
Orientation (")
Orientation (")
Orientation (")
Orientation (*)
p 001, huster--useed pentrultttion test
Figure 3: The distribution of behavioral variability and estimated resource J. (A) WM variability grouped by the context. (B) The resource distribution estimated by a variable precision model. The shaded areas represent the clusters showing significant differences in resource between the two delay durations (permutation test, < .001). Color shades and error bars represent + s.e.m.
We view WM resource as quantities that can be distributed improve encoding and maintenance quality. From behavioral data, we estimated the theoretical resource quantity using Fisher information by the Variable Precision (VP) model (Va den Berg et al., 2012). In this model, the magnitude of memory error is associated with WM resource J. We extended the VP model by allowing the resource to vary as a function of orientation, and quantified how the resulting resource profiles changed systematically across contexts and delays (model details and parameter tests in A.1.7

---

We fitted the VP model to individual data in different contexts and delays, and then we treated the best-fit resource function (() as the estimated distribution of WM resource (Figure 33). The estimated resource roughly followed the environmental distribution of orientations, peaking cardinal orientations. Critically, rewards affected how WM resource evolved across delay. When reward was uniform (baseline context), resource declined over time for diagonal orientations but were maintained well for cardinal ones, reflecting preferential maintenance for more frequent stimuli. In contrast, in the reward context, the cardinal orientations (now associated with lower reward) have their resource decreased under longer delay whereas the resource for the diagonal orientations (associated with higher reward) stable. These results confirm a joint effect of stimulus prior and rewards on WM resource allocation.
Efficient coding for utility maximization in working memory
3.1 Previous studies on efficient coding of perception
The theory of efficient coding posits that perceptual system should allocate more encoding resource with respect to the statistical structure of the environment by allocating more resource more common stimuli [Barlow et al.) 1961]. Recent studies on efficient coding have derived optimal allocation of neural resource quantified as Fisher information (Wei and Stocker 2015 Hahn and Weil
2024, Morais and Pillow (2018). Under this framework, resource allocation can be formalized as the objective to minimize overall expected loss defined as:
L -= f p(0) [m(m) 01" de
(1)
Here m is the sensory measurement of the stimuli, and p is an exponent term that specifies the loss function. Extending the theoretical framework developed in Wei and Stocker (2015). Morais and with neural resource (Fisher information J), and identified the lower bound of the expected loss in Eq above as L = f p(0) {()) dd. Under the constraint of total resource f (()) do < C, where C' represents total resource and /3 specifies the non-linearity in the constraint, it was shown that the optimal resource allocation is proportional to the stimulus prior distribution raised to a power q. a power-law efficient code [Morais and Pillow 2018]
Pillow 2018) linked the estimation
Jopt x p(0)
3.2 Extension of power-law efficient code to utility maximization
Here, we extended this framework to tasks in which stimuli 0 carry context-dependent rewards. Similar ideas have been proposed for perceptual encoding [Schaffner et al. [023]. We shift the objective from minimizing estimation errors to maximizing expected utility. In our tasks, reward decreases monotonically with estimation error. Therefore, the expected reward can be written as:
f p(0) (00) B(d(O, 0)] do
R
where uB) is the value function determining the orientation-reward association, d(ê,0) is reward discounting function-a decreasing function of error. A simple choice is d(0,0) 0) = B cl0 0. where B > O can be considered as a baseline payoff (e.g., the subject payment), and c > 0 dictates how rapidly rewards declines with estimation error. Given that and are constants, maximizing reward is equivalent to minimizing the loss:
/ p(0) (0)) 0 dB
L L=
This objective has a form similar to Eq Thus, the power-law efficient coding can be extended for utility maximization as:
Jom x (p()) x0)]a
(2)
That is, the optimal resource is proportional to the expected value of the stimulus raised to a power. In the reward context of our behavioral experiments, reward increased linearly with the diagonality of
5

---

orientations, opposite to the trend in natural stimulus statistics. Consequently, the optimal resource allocation, and thus the variability perceptual memory reports, would be flattened relative to the baseline condition where reward is uniform (Figure A These predictions are in line with our behavioral results (Figure
A Optimal resource allocation
Theory:
Human data:
C
Estimated resource allocation
Theory
Human
RNN
Resource (J)
Efficient coding
wtlliy maxiniization
Resource (J)
Reploted from Figure 3B
Obilque effect
am amplitude
0
45
90
135
180
45
90
135
180
Orientation (*)
Orientation
(*)
Delay
Delay
Delay
p(0)
[p(0) v(0)}
p 2000
Basaline context
Reward context
Figure 4: Utility maximization extension of efficient coding theory. (A) Optimal resource for baseline context (Japs X p(0))), and reward context (Jepi [p(0) v(0)]"). Here set = 2 for illustration. (B) Resource allocation estimated from the behavioral data (5-sec delay) was consistent with the theoretical prediction (A). The shaded gray areas represent the clusters showing significant differences resource between the two contexts (permutation test, p < .001). (C) Left: The optimal resource allocation predicts a stronger oblique effect in the baseline than in the reward context. In addition, the increment of the oblique effect over time is more rapid in the baseline context. These predictions consistent with the behavioral data (middle) and the RNNs (right). Color shade error bar as the standard error.
3.3 Considerations on the time domain
One finding in the behavioral data was that, in the baseline context, the oblique effect became stronger with longer delay. This indicates that less frequent stimuli are not only encoded with lower precision, but are also harder to maintain over time. Here, we extend the power-law efficient code to incorporate time domain, allowing resource to vary with delay. The objective to optimize J(0, t), the resource distribution over orientations and time by minimizing the loss function:
[. p(0) (0, -) d0 dt
subject to the resource constraints summed over time:
('' 1(0 do
Let be the Lagrange multiplier. The full Lagrangian is:
C c=
[()) (N(O (()))((00))) 1(0,1) do di + > (c- 1-¹ 1(0,1) ao ai
Solving for J(0,t) t) yields:
(0 x p())
Importantly, even when incorporating the time domain, the optimal distribution of resource Jopt depends only on stimulus distribution p(0) regardless of time, implying fixed strategy throughout the memory delay. As a consequence, the difference in resource allocated between frequent (cardinal orientations) versus infrequent (diagonal orientations) stimuli will accumulate over time, producing a stronger oblique effect with longer delays, consistent with our empirical results (Figure in bascline context). This stable strategy can also be applied to utility maximization in which the optimal solution shared the same form Eq.
4 Recurrent neural network
The neural dynamics of WM has been extensively studied and modeled by recurrent neural networks (RNNs) with attractor dynamics that hold memory content in the presence of neural noise [Amit and

---

Bouchacourt and Buschman 2019 Compte et al. 2000 Wang 1999 2021 Wimmer al,, 2014, Acebee et al. [022] Here, we investigated whether and how biologically plausible circuit architecture RNNs, when combined with task-oriented and simple learning rules, can implement efficient and adaptive allocation of WM resource maximize expected reward, and how their behaviors compared to human performance!
Brunel 1997
4.1 RNN setup
We trained RNNs perform the same WM delayed-estimateee task as humans (Figure (E). To systematically investigate the effect prior and reward, we trained the RNNs a 2x2 design (Figure 5A). For the prior, input orientations were sampled either from uniform distribution (environmental) distribution, yielding the Uniform RNN and Environmental RNN, respectively. For the reward, networks were trained to maximize reward under contexts (implemented by the loss functions): a baseline context with uniform reward across orientations and a reward context where reward increased with orientation diagonality.
natural
Architecture
The input orientation 9 E (0, was encoded by 32 neurons with circular normal tuming function xx))] which were later normalized to [0, 1]. The centers of the encoding tuning functions o; evenly tiled the orientation space [0, rT. On each trial, an orientation stimulus was presented for 0.5s, followed by 5s memory delay (matched the human WM task) during which neural activity was maintained through recurrent connections among the neurons governed by Wree-
u4 exp[Fin cos (2 (2)(
The activity of the recurrent layer was updated based on the equation:
r(t + 1) a) r(t) + a tanh(r(t) u(0) a E(t)
where set as 0.2. The neural noise added each time step (per 20 ms) as E(t) NO, I), with a magnitude . = 0.5.
The outputs of the 32 response neurons are linear readout from the recurrent r()) activity {()
W Wout r()). Each output unit (() corresponded an orientation centered the same preferred angle 6; as its encoding counterpart, allowing the estimated orientation to be decoded via population vector averaging.
Loss functions and training
The RNNs were trained to minimize the loss function
L T Σ ((0) [|a(((]|
1 + Aact T E1r01) +
Au ||ore ||
where || denotes the squared estimation mated orientation at time step t. The total time steps per trial is T. The function v(f) specifies the stimulus-reward association. Under the baseline context, reward was uniform across orientations so this term was negligible, and was set as v(f) = 1. Under the reward context, reward increased with orientation diagonality (Figure 5 A), computed as:
between the stimulus orientation @ and esti-
π 2 m0) 0 mod?
d(0)
_ 4
|m(0) F.. v()) = 0.5 +
d(O) x/4
This weighting makes errors on more rewarded orientations contribute more strongly to the loss. In addition to the estimation error and reward, we imposed L2 regularization recurrent activities (Auet. = 10 10) and weights (A., = 10 to mimic biological resource constraints. In addition to different reward contexts, the networks were also trained under different stimulus priors by sampling the orientation 0 either from a uniform or from the environmental prior during the training (Figure 5A). We trained the networks using the Adams optimizer implemented in Py Porco with learning rate of 10-3, Each network reported here was trained with 15 random initializations through 2000 epochs with batch size as 256. After training, each network was tested with orientations uniformly sampled from 0" to 180 in steps of 2 with 300 repetitions per orientation, and we reported the results averaged over all the initializations.
Scripts available at htthttps:///ggg httpttthuu..omm//cetette wwwggiggii/nnnnnnnn.. Yang- 177/ UtilityMax.e.....o. git

---

4.2 RNNs results
Reward policy
Reward policy
Probabillyy
Stmmuuss distribution
Sumulus prior distribution
Natural prer Uniterm prior
Varibillty
Baseline context
Reward context
Delay
UnirUmmoon prio
Baseline context
Reward context
natural
Variablity Vartablitty
Os
Natural prier Umiom
J Blus 19
1s
Onentation (*)
Ostimulus
36
RNN
5s
Blas (4)
0
45
135
180
45
60 135 160
Orientation (*)
Orientation (*)
update weights
Aouputs
Transfer learning- frozen recurred weights
E
Transfer learning frozen readouf weights
Reward policy
uniform- - natural
Dacoline - reward
unifom natural
CAIII (1
Daasline reward
Reward
4 6
baseline reward
Oriontation (*)
Figure 5: RNNs architecture, and their variability and bias patterns. (A) RNN architecture, training stimulus distribution (uniform and natural distribution) and loss functions (reward and baseline context). (B) WM variability from RNNs trained with each stimulus distribution context condition. (C) Bias of RNNs trained with each stimulus distribution x context condition. (D) Transfer learning results when we froze recurrent weights trained in uniform distribution + baseline context, and retrained the readout layer in the new conditions. (E) Transfer learning results when we froze readout layer trained in uniform distribution baseline context, and retrained the recurrent weights in the new conditions. Only the results of variability are shown here. Each line denotes a delay length after the stimulus presentation duration. Color shades represent the standard error.
We trained the RNNs under different stimulus priors and reward contexts. Bias and variability were quantified using the same procedure as for human data. RNNs trained with the natural stimulus distribution and uniform reward (baseline context) showed the oblique effect: memory variability was lower for cardinal than diagonal orientations (Figure natural distribution baseline context), consistent with prior work using biased training stimuli in RNNs (Xiong and Weil 2022, Ye et al.) 2025 Eissa and Kilpatrick] 2023]. We found that with longer delays, variability increased more for infrequent (diagonal) orientations, amplifying the oblique effect, mirroring the behavioral data in the baseline context (Figure B)..
When RNNs were trained with natural prior but with reward policy favoring diagonal orientations, higher rewards stabilized these orientations by reducing their variability (Figure[3B natural distribution + reward context). Because the reward policy opposed the natural stimulus prior, their joint influences led to a weakened oblique effect. These RNN patterns are in line with efficient, dtility-maimmiiing allocation of WM resource (Figure AC), and resemble the behavioral results especially in the longer (5s) delay, where the oblique effect was significantly attenuated in the reward context relative to the baseline context (Figure
We also trained RNNs with uniform stimulus distribution under both contexts. As expected, when both prior and reward were uniform 31 uniform distribution + baseline context), variability was flat across orientations and simply increased over time due to the accumulated neural noise. Although human data did not show as strong reward effect, the results from uniform prior and reward context isolated the influence reward on memory variability (Figure 5B uniform distribution + reward context). In this case, memory variability lower for higher-reward orientations (diagonal) than lower-reward orientations (cardinal), and this pattern grew with delay.
While the RNNs reproduced clear effects of stimulus prior and reward on WM variability, and consistent with the human data, the bias patterns were less systematic (Figure5C)) A slight repulsion appeared in the RNNs trained with uniform prior under the reward context; however, this pattern likely reflects an attraction toward the high-reward orientations rather than the repulsion as in the previous studies on orientation Girshick et al. (201) Wei and Stocker [2017]. For RNNs trained with natural stimulus distribution with baseline context, we did not observe clear repulsive bias (from cardinal) as seen in humans, and there was trend of shifting towards more frequent stimuli.
8

---

4.3 Robustness and generalization
We performed systematic testing of the RNNs by varying network size (32 vs. 128 units), the internal recurrett noise level, the tuning centers of the encoding neurons, and regularization terms (details in A.2.4). These analyses showed that the effects of reward and prior on working memory variability were robust across architectures and configurations.
To address the concern that RNNs might simply memorize onput mappings, we trained the RNNs with a delay up to 5 sec, and showed that they generalized to longer (untrained) time points (7 sec) while maintaining the robust prior/reward effects (see Delay 7s in Figure These results confirmed that the models relied on dynamic memory representations, not static pattern matching.
4.4 Dissecting the contributions of recurrent dynamics
To investigate the unique contributions of the recurrent layer and readout layer, we performed two sets of transfer learning experiments to transfer across (i) prior stimulus distribution and (ii) the contexts, while freezing either the recurrent or the readout layer (details in A.2.2).
We first trained the RNNs under the uniform + baseline condition. In the first experiment, we then froze the recurrent weights, and retrained the readout layer in the (i) natural baseline (ii) uniform + reward condition. We observed a weaker effect from both prior and reward (Figure 5D): The variability patterns remained relatively flat across orientations compared to the RNNs fully trained in the new conditions (Figure BB. In the second experiment, we instead froze the readout layer, and retrained the recurrent layer. We found that the variability patterns (Figure 5):) were more similar to the RNNs fully trained in the new conditions. Therefore, the recurrent dynamics plays a more important role in allowing the RNNs to adapt to different stimulus distributions or reward policies.
4.5 Input noise and bias
While the bias was less systematic in our RNNs, there was weak but noticeable attraction bias toward the high prior or high reward stimuli (Figure The "attraction to prior" effect has been found in some WM studies that manipulated stimulus prior distributions [Panichello et al. 2019 Honig et al. 2020), and is generally in line with the Bayesian theory that an ideal observer would exhibit attraction to the prior [Knill and Pouget (2004) Ma 2019). The repulsive (from cardinal) bias often observed in orientation estimation (and other domains) is counterintuitive and interest in recent development efficient coding (Wei and Stocker 2015 2017, Hahn and Wei 2024). One possible account is that repulsion arises in an early encoding stage, where less frequent stimuli are represented with greater noise, preceding the recurrent network we modeled here. Thus, trained variant of RNNs where we injected input noise to the stimuli, with magnitude varying inversely with the environmental prior (stronger noise for more diagonal orientations) during training [Gu et al. 2025 The model exhibited clear repulsive bias away from the cardinals (Figure 6 and details in A.2.3)
Natural prior
Baseline context
Natural prior Reward context
Variability
Bias
Variability
Blas
Degree (*)
2
135
180
45
90
135
180
0
45
135
180
Orientation (*)
Orientation
Orientation (*)
Delay 06.
15
38
5s
Figure 6: RNNs variability and bias when trained with non-homogeneous input noise. Larger input noise near diagonal orientations produced a repulsive bias.
5 Limitations
While a repulsive bias could emerge when inhomogeneous input noise was introduced, such im- plementation does not explain how such encoding could learned by neural networks. more complete account may require training hicrarchical model, multi-layer RNNs (Yang et al. 2024]

---

with dedicated trainable encoding and maintenance stage to capture how the repulsive bias emerge through learning.
In this study, we used gradient-descent and backpropagation through time as the learning algorithms and showed that RNNs can learn to efficiently allocate resource for utility. Future studies can explore whether the same model behaviors emerge under more biologically plausible learning rules [Miconi 2017 Bellec et al. 2020).
Discussions
Using novel behavioral paradigm, we demonstrated that reward influences the precision and temporal dynamics of visual WM. Once stimulus-reward associations were learned, memory representations for high-reward stimuli were maintained with greater stability over time, reflected in the lower memory variability. These findings contrast with previous studies reporting null or mixed effects of monetary reward on WM (Van den Berg et al. [2023] [Brissenden et al.) 2023 Manga et al.) 2020 Weiss) jet al., 2025). The critical difference may lie in how the reward was manipulated: previous studies varied reward cues randomly across trials, whereas our design established consistent stimulus- reward associations that had to be Iearned. Treating reward as a stable feature of the environment may elicit more robust value-driven influence on WM, as in natural environments where reward contingencies are learned through experience.
By treating stimulus reward associations as an integral part of the environment, we extended efficient coding theory (Wei and Stocker] 2015 Morais and Pillow 2018 from minimizing estimation errors to maximizing expected utility, and further incorporated the temporal dimension for WM. The optimal resource allocation respects stimulus probability and associated reward jointly, and this optimal strategy remains stable over time. Importantly, both human behaviors and RNNs aligned with these theoretical predictions: (1) showing reduced oblique effect when diagonal orientations carried higher rewards, and (2) the effect of natural stimulus prior and the reward both accumulated over time. These converging results suggest that optimizing resource allocation with respect to utility, beyond merely minimizing estimation error, is fundamental component of adaptive behavior. To do so, organisms efficiently allocate WM resource to stabilize highly rewarding WM representations over time.
Previous studies on efficient coding jointly explained perceptual bias and discrimination threshold (Wei and Stocker 2015 2017], and suggested a coupling between the two. Our normative model and behavioral data diverged from this view: In extending efficient coding theory to the temporal domain and reward maximization, we focused on linking neural resource J with variability, without specifying the direction of bias. In terms of behavioral results, while variability increased over time, bias remained relatively stable, aligning with previous WM studies on orientations [Tomié et al., 2025 Shin et al. 2017]. The temporal dissociation between them may indicate a distinction between perceptual encoding and memory maintenance. Two factors may contribute this dissociation: First, this interpretation is consistent with findings that in human brains, WM engages neural populations that go beyond initial sensory encoding [Christophel et al.) 2017 Li et al. 2021] Li and Curtis 2023 Li al. 2025). While encoding depends on sensory areas with fixed tuning curves, WM further involves higher-level regions including parietal and prefrontal cortices, where neurons exhibit mixed and flexible selectivity (Rigotti et al.) 2013 Fusi et al., 2016). The dynamics of the higher-level regions may differ from the original encoding structure. Second, recent theoretical work (Hahn) and Wei 2024] suggests that bias arises from two opposing forces: a repulsive component during encoding and an attractive component during Bayesian decoding. These components can coexist, and if both strengthen over the delay, their effects may offset each other, resulting in a stable overall bias.
To conclude, the present study provides empirical evidence that humans allocate WM resource according to the expected utility from the learned stimulus rewaard associations. These behaviors consistent with an efficient coding framework in which the objective is to maximize individual utility. Notably, similar patterns emerge in pollogically-plassihhe RNN trained with simple learning rule to maximize expected reward. Together, our behavioral results, neural network models, and theoretical framework revealed that modulations in memory variability serve the primary mechanism through which the WM system allocates resource based on stimulus statistics and reward. Beyond their relevance to biological systems, our findings indicate that designing artificial agents aligned with human adaptive behavior requires integrating motivational and environmental factors that support utility maximization, rather than focusing solely on minimizing errors.
10

---

Acknowledgments and Disclosure of Funding
The authors thank the anonymous reviewers for their constructive feedback during the review process.
References
Daniel Amit and Nicolas Brunel. Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral cortex (New York, NY: 1991), 7(3):237-252, 1997. doi: 10.1099//error/333333..
Stuart Appelle. Perception and discrimination function of stimulus orientation: the" oblique effect" in man and animals. Psychological bulletin, 78(4):266, 1972. doi: 10.1037/h0031177.
Alan Baddeley. Working memory: looking back and looking forward. Nature reviews neuroscience, 4(10): $29-839. 2003. doi: 10.1038//rn120..
Horace B Barlow et al. Possible principles underlying the transformation of sensory messages. Sensory communication, 1001):217-23,, 1961.
Paul M Bays. Noise neural populations accounts for errors in working memory. Journal of Neuroscience, 34 (10):3632-306 2014, doi: 10. (0115//EUUOSSCC.224 13.2014.
Paul Bays, Sebastian Schneegans, Wei Ji Ma. and Timothy Brady. Representation and computation in visual working memory. Nature Human Behaviour, (6):1016-1034 2024, doi:
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj. Robert Legenstein, and Wolfgang Maass. solution to learning dilemma for recurrent networks spiking neurons. Nature communications, 11(1):3625, 2020, doi: 10. 10388/41466-0000 1700-773--
Flora Bouchacourt and Timothy J Buschman. A flexible model of working memory. Neuron, 103(1):147-16)) 2019. doi: 10. (016//..euron.20111
James A Brissenden, Tyler J Adkins, Yu Ting Hsu, and Taraz G Lee. Reward influences the allocation but not the availability of resources in visual working memory. Journal of Experimental Psychology: General, 152 (7):1825, 2023. doi: 10.1037/xge00001370
Thomas B Christophel, P Christiaan Klink, Bernhard Spitzer, Pieter R Roelfsema, and John-Dylan Haynes. The distributed nature of working memory. Trends in cognitive sciences, 21(2):1 2017. doi: 10.1016/f. 12.007.
Anne GE Collins and Michael Frank. How much of reinforcement learning is working memory, not reinforce- ment learning? behavioral, computational. and neurogenetic analysis. European Journal Neuroscience, 35(7):1024-1035, 2012. doi: 10.1111/f.11e 1/i.111/1111----1 956992018111111111 1.09900..
Albert Compte, Nicolas Brunel, Patricia S Goldman-Rake Rakic, and Xiao-Jing Wang. Synaptic mechanisms and network dynamics underlying spatial working memory in cortical network model. Cerebral cortex, 10(9): 910-923, 2000. doi: 10. 10.1093/cercor/10.9
Meredyth Daneman and Patricia A Carpenter. Individual differences in working memory and reading. Journal of verbal learning and verbal behavior, 4664 1980. doi: (1101680002-5577708))00111
Vincent De Gardelle, Sid Kouider, and Jerome Sackur. An oblique illusion modulated by visibility: Non- monotonic sensory integration in orientation processing. Journal of vision, 10( (0(10))6-6, 2010. doi: 10.1 10.10.6.
Blaire Dube, Stephen M Emrich, and Naseem Al- Al-Aidroos. More than a filter: Feature-based attention regulates the distribution of visual working memory resources. Journal of Experimental Psychology: Human Perception and Performance, 43 1841,, 2017. doi: 10.1037/xhp0000428.
Daniel B Ehrlich and John D Murray. Geometry of neural computation unifies working memory and plan- ning. Proceedings the National Academy of Sciences, 119(77):c111610011 2022. doi: 10. 1073/as/ss 2115610119.
Tahra L Eissa and Zachary P Kilpatrick. Learning efficient representations of environmental priors in working memory. PLOS Computational Biology, 19(1 1001 1622, 2023. doi: 10.1371/joummll.pe 101 1622.
Stephen M Emrich, Holly A Lockhart, and Naseem Al-Aidroos. Attention mediates the flexible allocation of visual working memory resources. Journal of Experimental Psychology: Human Perception and Performance, 4454):1454, 2017. doi: 10.1037/xhp0000399.

---

Jose M Esnaola-Accbes, Alex Roxin, and Klaus Wimmer. Flexible integration of continuous sensory evidence in perceptual estimation tasks. Proceedings of the National Academy of Sciences, (59(445):2224441111 2022. doi: 10. 10.1073/pnas.2 2214441119.
Stefano Fusi, Earl Miller, and Mattia Rigotti. Why neurons mix: high dimensionality for higher cognition. Current opinion in neurobiology, 37:66-74, 2016. doi: co0.6//..ob.2216..00 .010..
Ahna R Girshick, Michael S Landy, and Ecro P Simoncelli. Cardinal rules: visual orientation perception reflects knowledge of environmental statisties. Nature neuroscience, 14(7):926-95) 9322, 2011. doi: 10.1038/nn.288 2831.
Hyunwoo Gu, Joonwon Lee, Sungje Kim, Jacscob Lim, Hyang- Jung Lec, Heeseung Lec, Min Jin Choe, Dong- gyu Yoo, Jun Hwan Joshua Ryu, Sukbin Lim, al. Attractor dynamics of working memory explain concurrent evolution of spimulus-specific and doi: 10. 101 10..1066j..eeeeee.ee neuron.2025.5 07.003.
biases in visual estimation. Neuron, 2025.
Michael Hahn and Xue-Xin Wei. A unifying theory explains seemingly contradictory biases in perceptual estimation. Nature Neuroscience, 27(4):793-804, 2024. doi: 10.10088445993002-0555444.
Maija Honig, Wei Ji Ma, and Daryl Fougnie. Humans incorporate trial-to-trial working memory uncertainty into rewarded decisions. Proceedings of the National Academy of Sciences, 117(15):331-8397,, 2020. doi: 1t..0////aa..111111
Zuzanna Klyszejko, Masih Rahmati, and Clayton E Curtis. Attentional priority determines working memory precision. Vision research, 105:70-76, 2014. doi: visre//..srss.01111.1 09.002.
David C Knill and Alexandre Pouget. The bayesian brain: the role of uncertainty in neural coding and computation. TRENDS Neurosciences, 27(12):112-199 2004. doi: 10.1016//.tins.222 20..22004..000000
Hsin-Hung and Clayton Curtis. Neural population dynamics human working memory. Current Biology. 33(17):3775-33) 3784, 2023, doi: 10. 10.10166f.cbb2203300..666
Hsin-Hung Li, Thomas C Sprague, Aspen H Yoo, Wei Ji Ma, and Clayton E Curtis. Joint representation of working memory and uncertainty in human cortex. Newron, 109(22):399-3371, 2021. doi: 10.1016/j.nnuroo 2021.08.022.
Hsin-Hung Li, Thomas C Sprague, Aspen Yoo, Wei Ji Ma, and Clayton E Curtis. Neural mechanisms of resource allocation in working memory. Science Advances, (1(5))ead88015 2025. doi: 10. 126/sciadv. adr8015.
Steven J Luck and Edward K Vogel. The capacity of visual working memory for features and conjunctions. Nature, 390(6657):279 2-1,, 1997. doi: 10.1038/36846.
Steven J Luck and Edward K Vogel. Visual working memory capacity: from psychophysics and neurobiology individual differences. Trends in cognitive sciences. 17(8)3391 400, 2013. doi: 10.1016/j.iies.2113..
Wei Ma. Bayesian decision models: A primer. Neuron, 104(1):164-175, 2019. doi: 10.10166..euumn.2019.00 037.
Wei Ji Ma, Masud Husain, and Paul M Bays. Changing concepts of working memory. Nature neuroscience, 17 (3):447335,, 2014. doi: 10.1038/mm.655.
Annamária Manga, Pál Vakli, and Zoltán Vidnyánszky. The influence of anticipated monetary incentives on visual working memory performance healthy younger and older adults. Scienrific Reports, 10(1))8817, 2020. doi: 10.
Thomas Miconi. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife, 6:c20899, 2017. doi: 10.7554/eLife.2088
Michael Morais and Jonathan W Pillow, Power-law efficient neural codes provide general link hetween perceptual bias and discriminability. Advances in Neural Information Processing Systems, 31, 2018
Matthew F Panichello, Brian DePasqualc, Jonathan W Pillow, and Timothy Buschman. Error-correcting dynam- ics in visual working memory. Nature communications, 10(1):3366, 2019. doi: 10.1038/541 167-0444467-99--- 11298-3.
Yoni Pertzov, Sanjay Manohar, and Masud Husain. Rapid forgetting results from competition over time between items in visual working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43 (4):528, 2017. doi: 10. 10.100//mm0000328.
12

---

Mattia Rigotti, Omri Barak, Melissa R Warden, Xiao-Jing Wang, Nathaniel D Daw. Earl K Miller, and Stefano Fusi. The importance of mixed selectivity in complex cognitive tasks. Nature, 497(7451):5555590 2013. doi: 1..1088/nture12166
Jonathan Schaffner, Sherry Dongqi Bao, Philippe N Tobler, Todd A Hare, and Rafael Polania. Sensory perception relies on tlness-maximizing codes. Nature Human Behaviour, 7(7):1335-115, 2023, doi: (0000885415522-22-05554444
Hongsup Shin, Qijia Zou, and Wei Ji Ma. The effects of delay duration on visual working memory for orientation. Journal of vision. 17(14):10 10, 2017. doi: 10.1167/17.14.10.
Heinz-Martin SüB, Klaus Oberauer, Werner Wittmann, Oliver Wilhelm, and Ralf Schulze. Working- memory capacity explains reasoning ability-and and a little bit more. Intelligence, 30(3):261-288, 2002. doi:
10.1016/S0160-200 60-2606(010666)) 300100-3.
Robert Taylor and Paul M Bays. Efficient coding in visual working memory accounts for Himmllus-specinie variations in recall. Journal of Neuroscience, 38(32):7132-7144 2018. doi: J0.1523/NEUROSCL.JJ 101 2018.
Ivan Tomic, Sebastian Schneegans, and Paul M Bays. Dynamics of variability and bias in working memory. PsyArXiv, 2025, doi: 10.31234hotiooly2byyy...
Ronald Van den Berg, Hongsup Shin, Wen-Chuang Chou, Ryan George, and Wei Ji Ma. Variability in encoding precision accounts for visual short-term memory limitations. Proceedings of the National Academy of Sciences, (1122):7780-8885 2012. doi: 10. (0.1073/pnas.] 1..77//pas..1114651000
Ronald Van den Berg, Edward Awh, and Wei Ji Ma. Factorial comparison of working memory models. Psychological review, 121(1):124, 2014, doi: 10. 10.3033//033523
Ronald Van den Berg, Qijia Zou, Yuhang Li, and Wei Ji Ma. No effect of monetary reward in visual working memory task. PloS one, 18( 18())e0028227, 2023. doi: 10. 10.1371 //journll.pone 1o0e0m5...one.0020255...
Xiao-Jing Wang. Synaptic basis of cortical persistent activity: the importance of amda receptors working memory. Journal of Neuroscience, 19(21):9587-966 9603, 1999. doi: LL52:/NEURRSSC..1222-0000 -99887.199.
Xiao -ing Wang. 50 years of mnemonic persistent activity: quo vadis? Trends in Neurosciences, 4:(11):888 902, 2021. doi: htt.://..n............
Xue-Xin Wei and Alan A Stocker. A bayesian observer model constrained by efficient coding explain anti- bayesian' percepts. Nature neuroscience, 18(10)11509-1517 2015. doi: 10. 1...00.n...05...
Xue-Xin Wei and Alan Stocker. Lawful relation between perceptual bias and discriminability. Proceedings of the National Academy of Sciences. 114(38):10244 10249, 2017. doi: 10.1077/pmns.1111151114
Béla Weiss, Annamária Manga, Ádám Nárai, Adél Bihari, Judit Zsuga, and Zoltán Vidnyánszky, Reward boosts cognitive control during working memory maintenance. Scientific Reports, 15(1):29322, 2025. doi:
https://doi.org: 10. 1588844198--05-0949999
Klaus Wimmer, Duane Q Nykamp, Christos Constantinidis, and Albert Compte. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nature neuroscience, 17(3): 431-439, 2014. doi: 10. 10.1038/nn.365.
Hua-Dong Xiong and Xue-Xin Wei. Optimal encoding of prior information in noisy working memory systems. In Conf. Cogn. Comput. Meeroosc.((CM)) 2022), pages 2022-1162, 2022. doi: 10.32470/CCN.2022.1 116-00 162--0.
Jun Yang, Hanqi Zhang, and Sukbin Lim. menooyy interactions via modular structure explain errors in visual working memory. Elife, 13:RP95160, 2024. doi: 10.7554/eLife.951660
Zeyuan Ye, Haoran Li, I Liang Tian, and Changsong Zhou. The effects of the post-delay epochs on working memory error reduction. PLOS Computational Biology, 1001:33,,33 2025. doi: 13. pebi. 1013083.
Aspen H Yoo, Zazanna Klyszejko, Clayton E Curtis, and Wei Ji Ma. Strategic allocation of working memory resource. Scientific reports, 611),16162, 2018. doi: 10. 1...0088/441998 1-98-018-3428--11
Weiwei Zhang and Steven J Luck. Discrete fixed-resolution representations in visual working memory. Nature, 453(7192)233--23, 2008 doi: 10.10088maturr006600
13

---

A Technical Appendices and Supplementary Material
A.1 Behavioral methods and results
A.1.1 Value learning trajectory
The value learning task followed the
association that, in the reward context, reward increased with the diagonality of the orientations: in the baseline context, all the orientations were associated with same reward (Figure (3). To visualize the learning trajectory, trials were biuned into 10 trials per bin based on their order (Figure 7 The percentage of subjects choosing the most diagonal stimuli in the reward context was consistently higher than in the haseline context after 40 trials of learning (FDR corrected paired t-test, P .01). Learning the reward context reached an asymptote (about 70% choosing the highest-reward stimuli) around 60 trials. In baseline context, the probability of choosing the most diagonal stimuli (now not high-reward) of all participants reached the chance level 25% after 70 trials of learning (FDR corrected one-sample t-tests, > .05).
Learning trajectory
P(Choose the most diagenal ilam)
Baseline context
Reward context
0.25
10 20 30
40 50 60 70 a0 90100
Figure 7: Learning trajectory in the value learning task.
A.1.2 Power analysis
We conducted power analysis based the effect size reported in the 1 in Schaffner et al. 2023 which investigated the effect of reward on perceptual encoding. Specifically, used the reported coethcient (Estimate -0.16, SE' 0.09) from Bayesian hierarchical linear regression model to approximate a standardized effect size (Cohen's f = 2.0). This effect size represents the magnitude of the reward-related difference between the reward and baseline contexts for oblique orientations. Based on the orientation bins, we applicd Bonferroni correction for multiple comparisons, resulting in corrected significance threshold of 0.01. Power analysis revealed that participants provided above 90% power detect an effect of reward in the oblique orientation.
A.1.3 Compensation information
Each subject was compensated at the rate of 12 dollars per hour for each session, and there up to 10-dollar bonus based on the points collected each day (context).
A.1.4
WM bias magnitude
We modeled the signed response error a periodic function of the stimulus orientation E [0 180"] using third-order Fourier series regression. For each context x delay condition, the model was defined as:
3
erfor (0) Bu + Σ [Bas cos(jo) + Ba., sin(jo)] +
which approximates the response error as a weighted sum of sinusoidal functions. Here the free parameters included fo as the intercept, and Begj and 8..j as the coefficients (weights). All parameters were estimated via ordinary least squares by minimizing the sum of squared difference between predicted er)or(O) (the bias) and observed error(o).
To quantify the overall magnitude of the bias, we compared the signed peaks within 0-15 (erfori) and within 135- 180 (erfor2), each determined by the maximum absolute value of the predicted bias that range. The resulting bias magnitude M was computed as
M erior: - erforz.
A positive bias magnitude represents a repulsive bias from the cardinal orientation, and a negative value denotes an attraction towards cardinals. If erfor; and erforz share the same sign, the corresponding value was excluded from group analyses.
14

---

A.1.5 WM variability
To isolates trial-by triatral variability independent of systematic, deentation-dependent bias, the predicted bias erfor(A) from Fourier series regression was subtracted from the response error to yield bias-removed residual:
error(0) = error(0) - erior(0),
We quantified the WM variability Var(O) within each condition (and orientation hin) as the standard deviation of the residual error(0), providing bias-removed response precision.
A.1.6 Quantify effects in WM variability
After obtaining bias-removed variability Var(ø), we quantified the effect of context x delay in variability by fitting sinusoidal function of the form:
w 180 + +b) Basever,
Var(0) Aver sin
where & denotes the orientation in degrees, controls the frequency, & the phase offset, Aver the amplitude, and Base the baseline level of variability which is constant across orientations. This formulation captures systematic modulations response variability across orientations (e.g.. oblique effects).
Model parameters (wc, 6, ABar, Base were estimated separately for each condition by minimizing the squared error between the observed and predicted variability:
mii (W.O.A.B)
Σ
[var(0))Var(())]
- var(o)]²
using global optimization approach with randomized initialization and bounded parameter ranges to ensure robust convergence.
Among the parameters, the amplitude Avar reflects the strength of modulation in variability from orientation. Therefore, we use Avar to capture the relative advantage error variability at cardinal orientations, where larger Aver indicate a higher difference in variability cardinal relative to oblique orientations, such as a stronger oblique effect.
We fitted each participant's 's variability results in each context delay condition with the sinusoidal function ahove, and applied repeated-measurrel ANOVA to investigate the effect of reward and context:
For Avar. the amplitude parameter (Figure 2B), there was significant interaction of context and delay (p < .05). and no main effect of context the cardinal orientations has lower variability compared to the oblique was affected by reward-orientation association in the WM longer delay.
.19) nor delay -20). The strength of the oblique effect- how much
For Basevar, the base level of the variability across orientations, there was no significant interaction of context and delay (p = .76), but a significant main effect of context (p = .04) and delay (p < .001).
For o, the peak location, there was no significant interaction of context and delay (p = .39), and no main effect of context (p .25) and delay (p .41). The peak was stable around the oblique orientations.
For w, the frequency, there was significant interaction of context and delay (p _57), and no main effect of context (p 113 and delay (p 22). The frequency was stable provide 90 periodicity.
A.1.7 WM resource allocation distribution
From the behavioral data, estimated the theoretical resource quantity Fisher information J. We adapted the Variable Precision (VP) model to memory error distributions under different contexts and delays [Van den Berg et al. 2012]. Memory errors (hias-removed) were modeled as samples from von Mises distributions p(c) von woo_ Mises(O, x). where e denotes the angular memory error and k is the concentration parameter. In the model, x was linked with the fisher information - WM resource parameter J by J = к Io(e)) where Ta(x) and (N) are the modified Bessel functions of the first kind (of order 0 and 1). Higher values produced narrower distributions. Although not our main focus, the VP model assumed that itself is a random variable
(e())
following gamma distribution J gamma ( 7). where J is the mean of the resource parameter J. and is the scale parameter of the Gamma distribution, controlling the variability in J across trials (Van den Berg al. 2012). To capture iieetattoo-deeendeeee resource allocation, we modeled the mean resource J as function of orientations:
J(0)- = A (1 - B |sin + D)D"
where A, B, & and were fitted free parameters, allowing J to vary across orientations.
15

---

For the resource distribution represented by the best-fitted resource function J(0) (1 B |20 + +))) Figure 3 showed that resource distributions exhibited stronger oblique effect (the W pattern) in the long delay condition the baseline context, but weaker oblique effect the long delay condition in the reward context (a shallower W). These effects are reflected in the parameter B. Indeed, we found significant interaction effect (context x delay, < .05) when testing the best-fitted B values. Here we describe the statistical results all the fitted parameters:
A
For A: The linear mixed-effects model showed no significant interaction effect (context x delay, = .27), and no significant main effect of context (p .27) or delay (p .28). The Bonferroni-corrccted one-sample t-test showed that A is significantly larger than (p < .001) all contexts and delays.
For B: The linear mixed-effects model showed a significant interaction effect (context x delay, p < .05), and a marginally significant effect for delay (p = -06) and no effect for context (p = .5). The Bonferroni corrected one-sample t-test showed that B is significantly larger than 0 (p < .001) in all contexts and delays.
For di The linear mixeddefffect model showed marginal interaction effect (context x delay, p = .66), and no significant main effect for delay (p- .22) context (p .59). The Bonferroni-corrected oae-se t-test showed that ø only significantly larger than in the delay 5 condition in both contexts, while not significantly different from 0 in the delay = s condition in either context.
For n: The linear mixed-effects model showed no significant interaction effect (context x delay, p = .25), but significant main effect of delay (p < .05) and no effect of context (p .37). The conferroni-correctt one-sample 1-test showed that is significantly larger than 0 (p < .001) in all contexts and delays.
A.2 RNN methods and results
A.2.1 Computing resource
Modeling training required approximately 10 minutes for each initialization on single T4 GPU for each setup. Therefore, the results of the main RNN experiment we reported in the paper (Figure 55A) took roughly 10 hours for model training.
A.2.2 Transfer learning
Additional details of the transfer learning experiments are described here: We conducted 2 transfer learning tasks (actoss contexts, and priors), and tested the contribution of 2 different layers (recurrent and readout layer) in the RNNs in adapting stimulus prior distribution and reward policy.
(1) Transfer across contexts (baseline - reward). We first trained 15 RNNs the uniform distribution baseline context fully with varying initializations, then (i) froze the recurrent weights and retrained the readout in the uniform distribution reward context; or (ii) froze the readout layer and retrained the recurrent weights in the uniform distribution +reward context. All the other configurations of the RNNs remained the same as those in the main text.
(2) Transfer across prior distributions (uniform - natural). We trained 15 RNNs under uniform distribution + baseline context with varying initializations. After training, we then (i) froze the recurrent weights and retrained only the readout weights in the natural distribution + baseline context; or (ii) froze the readout layer and retrained the recurrent weights in the natural distribution + baseline context. All the other configurations of the RNNs remained the same.
For bias, the transferred RNNs closely matched fully trained RNNs in the new conditions, when we froze and retrain either layer, indicating that both layer explains the effect of prior and reward policy in the bias. The results for the variability are described in the main text 5. E).
A.2.3 Input noise level
For this RNN variant, we added gaussian noise to the stimulus input during the training while keeping all the other configurations the same, We assumed that the recurrent neurons received a noisy version of the stimulus input. The noisy input 0 modeled as the true stimulus input do perturbed by Gaussian noise 0 ~ ~N((0,, ? (2n)) where we set 7 as 10 [Gu et al. 2025). This non-homogeneous input noise led to higher input noise for more diagonal orientations. This implementation was based on the assumption that the repulsive bias or oblique effect involves non-homogeneous encoding that occurred before the recurrent neurons. When calculating the estimation error in the loss function, the decoded orientation was compared with the true stimulus input 0p.
After adding the noise, the stimulus distribution for natural prior condition remained similar to the original stimulus distribution: therefore the RNN training results reflect the effects of both the natural stimulus distribution and input noise (Figure B). However, the uniform distribution was affected by adding these non- homogeneomu
16

---

A Uniform distribution
B
Natural distribution
Probability
Probability
Original input Noisy input
45 90 135180 Orientation (*)
45 90 135180 Orientation (*)
Figure 8: Stimulus distribution before and after adding the stimulus noise.
input noise, and we also observed decrcased variability at oblique orientations compared to the RNNs without input noise.
A.2.4 Robustness
Larger network.
We trained 15 large RNNs with 128 recurrent units (vs. 32 as we reported in the main paper) in each prior x context condition with varying initializations (Figure The variability of the large RNNs under larger network shared the same pattern with the smaller network, indicating the effect of reward and prior distribution. The average variability was smaller for larger RNNs, suggesting improved precision with larger networks. The magnitude the bias for larger RNNs was also small.
A Uniform Baseline context
Uniform + Reward context
B
Uniffrm + Baseline context
Uniform Reward context
Baseline
Delay Os
Jarsaiilty
Blos (7)
Delay 1s
Delay
Delay 5s
Delay 75
90
135
160
90
135
160
Orientation 160
Orientation (")
Orientation 1"]
Orientation
Natural Baseline context
Natural Reward context
Baseline
Varabiltty (*)
Reward
Delay Os
Delay 15
2
Bas
Bios
Delay 5s
90
160
0
135
45
90
135
160
90
135
180
Delay
Orientation
Orientation
Orlentation
Orientation (*)
Figure 9: The larger RNN (128 units) WM performance. (A) Variability patterns for larger RNNs reflected the impact of reward and prior stimulus distribution. (B) Bias patterns.
Alternative loss weightings.
We trained additional RNNs with larger L2 penalty recurrent weights and activations (10-2 vs. 10-4 reported in the main paper), while keeping other configurations the same (Figure TO). We observed stronger that, compared the weaker regularization, these models with stronger regularization exhibited modulations by stimulus prior or reward large differences variability between low- versus high- tigh-probabilltyy orientations, or between low- versus high-reward orientations. There was stronger bias towards the frequent and higherewarded orientations. Thus, more constrained working memory system allocates its capacity more selectively across stimuli.
Additionally, trained RNNs with L2 penalty recurrent weights and activations. These models exhibited behavior similar those trained with small (10-4) L2 penalties that are affected by both prior and reward conditions. These results suggest that resource- efficceent behavior can emerge from task optimization alone, even in the absence of hard-coded limitations, Thus, "limited resource" in working memory may reflect rational strategy to task goals rather than strict architectural bottlenecks.
Centers of the encoding tuning function
To investigate whether the results observed were related to the location of the orientation tuning function of the RNN neurons during stimulus encoding, trained collection of RNNs with slightly shifted tuning functions. For each of RNNs with varying initializations, the unit centers were initialized
centers: = (cenders + dcenter + e;) mod m.
where centers; are fixed center values that evenly spaced the orientation space, Pwenter is global phase shift uniformly sampled from (-5",57 5)). The +5 shift range corresponds approximately to spacing step
17

---

A
Unitorm Baseline context
Unitomm Reward context
B
Uniform Baseline context
Unitorm Reward context
Baseline
Varibiltyy (
Veriabiliy 4%
Delay 0:
Bias C?)
20
Deley 35
Delby St
20
-0
90
180
45
135
100
50 Orientatio (*)
135
180
45
90
135
160
Orientation
Orientation (*)
Orientation (*)
Natural Baseline context
Natural Reward context
Natural Baseline
Natural Reward context
Reward
Dalay 0s
Varibility
Bias e*
Delay 1s
20
20
Bias
Delay Ss
Delay 7s
45
90
135
180
45
90
135
100
45
90
135
180
45
90
135
160
Orientation (*)
Ortentation (*)
Ortentation
Orientation (*)
Figure 10: WM variability and bias for RNNs with larger L2 regularization on recurrent weights and activations.
(180% /32) between adjacent orientation-tune tuned units. The (0,0.55% is a small per-unit jitter. All other aspects were identical the original models. The resulting RNNs behaviors were the same what reported the main paper (Figure
A Uniform Baseline context
Uniform Reward context
B 0na T7
Uniform + Baseline context
Uniform + Reward context
Beseline
Delay
Verseiity
L1
Delay 1s
Delay 5s
Delay 75
180
45
135
135
180
Orientation
Ortentation
Natural Baseline context
Natural Reward context
Natural Baseline context
Natural Reward context
Reward
199999 13
Delay 1s
Dies
Delay
Delay
Delay 7s
90
180
46
135 160
45
1is
180
90
135
180
Orientation (*)
Orientation (")
Orientation (*)
Orientation (*)
Figure 11: WM performance for RNNs with the shifted centers of the tuning functions during perceptual encoding.
Internal noise level
To investigate the effect of internal noise, we trained and tested RNN under different levels of internal sensory noise levels (a = 0.1 0.9, the neural noise added to the recurrent activation per time step), as shown in Figure 12k. (I) When we matched the training and testing noise, the reported variability pattern holds consistent across different levels of noise. (II) When RNNs were tested lower noise than training. the reported variability pattern (Figure 55A) holds consistent and even stronger. For bias, there attractive blases toward the prior reward. (III) When RNNs were tested at higher noise than training, the oblique effect in variability broke down. Bias became more repulsive from prior or high-reward stimuli.
We further trained RNNs with varying levels of internal neural noise, where we randomly and uniformly selected internal noise level for each of the 2000 epochs during training. After training, these RNNs were tested with optimized for both prior and reward policy, with increasing effect when tested with higher sensory noise setting. For bias, we also observed amplified effect from prior and reward with higher testing noise. Compared the RNNs trained with a constant level of internal noise, these RNNs trained with varying noise showed more robust effect of prior and bias, preserving the patterns regardless of the testing noise.
constant noise levels (Figure B). The variability amplitude
18

---

Uniform baseline, Delay
Uniforrm reward, Delay
Natural baseline, Delay 5s
Natural reward, Delay 5s
A
0.8
0.8
0.9
Variabllity amplltude
0.9
0.8
Veriabiity amplitue
0.8
0.9
8 0.7
Test nolse
0.7
Test naise
0.7
23 0.4
0.4
0.4
0.4
0.3
0.3
0.3
313
685000000
500666600
2N0900000 00
Train nose
Train miie
Train naise
Train noise
Uniform baseline, Dalay
Uniform reward, Delay Ss
Natural + baseline, Delay 5s
Natural + reward, Delay 55
0.9
epusion
Bies Sangnttute (')
0.6
0.8
0.5
0
25
0 0.7
0.6
25
Blas magnitude
nese
0.6
0.5
0.7
25
Blss magnitude
Test noise
0.7
0.6
0.8
25
0.4
0.2
-25
0.2
-25
0.3
0.2
0.
0.1
-N49000000
m 1ine 68 660006000
095006806
30006660
Train noise
Go0o06608
B
Variability amplitude (?)
Bias magnitude (*)
Uniform Baseline
1
Uniform Baseline
2.5
Natural Baseline
Uniform Reward
Natural Baseline
Uniform + Reward
0.0
Reward
-
66
Testing noise
Testing noise
The effect of training and testing internal noise in RNNs WM. (A) RNNs variability amplitude and bias magniude under different training and testing of internal noise level. For the variability, the red colors indicate an oblique (higher variability at diagonal orientations) while the blue colors indicate a pattern opposite the oblique effect. For the bias, green colors indicate an attraction to the cardinal orientations while the pink colors indicate repulsion. (B) The variability amplitude and bias magnitude for the RNNs trained with varying internal noise and tested with different level of internal noises.
Figure 12:
19

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly reflect the work the section "Joint effects of prior and reward on working memory dynamics", "Efficient coding of working memory" and "RNN" parts.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The ahstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can he expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Refer to the Limitations section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper limitations, but those are not discussed in the paper.
The authors are encouraged create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should he articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to- text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection,
outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We included the assumption the "Efficient coding of working memory" section with specific formula and reasoning, the behavioral and RNN results can be considered as the supporting evidence.
Guidelines:
The answer NA means that the paper does not include theoretical results.
20

---

All the theorems, formulas, and proofs in the paper should be numbered and cross tross-reference..
All assumptions should he clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We included the detailed information for behavioral experiments subsection "Behav- ioral tasks" and details of RNN in "RNN setup". Some details the appendix.
Guidelines:
The answer means that the paper does not include experiments.
the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on contribution, reproducibility can be accomplished various ways. For example, if the contrihution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others replicate the model with the same dataset, or provide access the model. general. releasing code and data often one good way to accomplish this, reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the rescarch performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility. which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source source models, it may he that access to the model is limited in some way (e.g.. registered users), but it should be possible for other rescarchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does paper provide open access to the data code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The preprocessed data and code will be shared at https://github. com/
Qingging Yang- 177/wm UtilityMax..tth
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines htttpe://nipps.cc/ubbbbs
guides/Codde Cowe/oooeeeeeaee ioddeSubaissioooPliooo. for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
21

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines htttps://nips.cc/puut for more details.
Buidee//DodeSuumiissioooPPo
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release amonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental sefting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary understand results?
Answer: [Yes]
Justification:
We included training details of RNN in "RNN setup".
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error hars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: The bars the behavioral results and RNN results are all using the standard errors. We reported statistical tests and results.
Guidelines:
The answer means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for caleulating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 96% CI. if the hypothesis of Normality of errors
a 2-sigma error than state that they not verified.
For asymmetric distributions, authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: The model fitting the WM resource allocation and RNN training and testing utlizes GPU and could also be adapted to CPU, since the computing resources required are limited.
Guidelines:
22

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Fthies
Answer:
[Yes]
Justification:
The authors confirm the research conform with the code of ethics.
Guidelines:
The answer means that the authors have not reviewed the NeurIPS Code of Fthics.
the authors answer No, they should explain special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., if there is a special considcration duc to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The work more thcoretical cognitive neuroscience work for working memory, which does not have direct impact on the societal components.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- ment technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimatc point out that improvement in the quality of generative models could used to generate deepfakes for disinformation. On the other hand, it not needed point out a generic algorithm optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms monitor- ing misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer:[NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
23

---

Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled of the model, for example by requiring that adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: This paper properly cited the relevant original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset is used and, possible, include a URL.
name of license (e.g.. CC-BY 4.0) should be included each asset. scraped data from particular source (c.g., website), the copyright and terms of service of that source should provided.
assets are released, license, copyright information, and terms of use the package should be provided. For popular datasets, wwwppppppppcoooooo thcode. www.coo//caa/oc com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: the details the shared scripts are documented the README.nd.
Guidelines:
The answer NA means that the paper does release
Researchers should communicate the details of the dataset/code/moodl as part of their sub- missions structured templates. This includes details about training. license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an amonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: We included the instruction of the experiment in the "Behavioral tasks" part, and the compensation information the appendix.
Guidelines:
The answer NA means that paper does not involve crowdsoureing nor research with human subjects.
Including this information in supplemental material is fine, but main contribution of the paper involves human subjects, then as much detail as possible should he included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [Yes]
Justification:
The experiment is IRB approved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere to the NeurlPS Code of Ethics the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review,
Declaration LLM usage
Does the paper describe the usage of LLMSs if it an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method and algorithm development does not involve LLMs as original compo- nents.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, or non-standard components.
Please refer to our LLM policy (https://eeur) /ttp://////ru...eeeeeeeee Cct.Cooeeeeeecce/22225555 2LMM for what should or should not be described.
25


Reviews:
[{"review_id": "8ZlZwtIhxE", "reviewer": "Reviewer_Y8u3", "summary": "The manuscript investigates whether humans can efficiently allocate their working memory (WM) resources to maximize expected reward, and whether RNNs trained under similar constraints exhibit the same behavior. \n\nBehavioral side: 14 adult participants first learned stimulus–reward mappings and then performed an orientation-recall task after 1s or 5s delays. When rewards were uniform, memory precision followed the classical oblique effect (better for cardinals). When high rewards were assigned to oblique orientations, that pattern gradually inverted over the delay, indicating value-guided WM maintenance.\n\nComputational side: RNNs (32 units) trained with BPTT yield results that resemble those from human behavioral data.", "questions": "The paper assumes participants have fully learnt the reward schedule after about 100 trials. However, the asymptotic performance is only shown descriptively. Could you provide model-based learning-curve fits and exclude incomplete learners?\n\nTraining via BPTT is hard to accomplish in a biological system. Can you obtain similar results with frozen recurrent weights or without unfolding the computational graph in time (or some other non-BPTT training)?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nWhether WM reflects learned value, not just attention, is still debated; the paper provides relevant behavioural evidence as well as a modeling framework.\n\nExperimental design is clever. By making reward oppose the natural orientation prior, the design neatly teases apart prior vs. value effects.\n\nCombining data, theory and modeling with RNNs helps triangulate the mechanism and increases the work’s breadth.\n\nVariability–bias decomposition, mixed-effects modelling and VP fits are appropriate and clearly reported.\n\n\nWeaknesses:\n\nSmall sample size (14 subjects) limits statistical power.\n\nReward effects appear only in variability, not bias. The discussion speculates about encoding vs. maintenance, but the normative framework itself ignores bias.\n\nOnly one tiny architecture is reported. It is unclear whether the qualitative results survive with larger networks, different noise levels, or alternative loss weightings.", "was_revised": true, "final_justification": "Solid work. Authors have addressed my comments, although (as another reviewer pointed out) it is a bit difficult to judge all the updates in this format (without seeing detailed results implemented in pdf).", "rebuttal": "We thank the reviewers for their insightful comments. We are especially encouraged by the recognition of our study’s key strengths, including the clever experimental design and the integration of behavior and modeling. In response to the feedback, we conducted a power analysis, expanded the discussion on bias, and performed systematic tests examining the role of RNN architecture, noise levels, loss weightings, and transfer learning. We believe that these additions have strengthened our manuscript. Below, we address each point in detail.\n\n**1. Statistical power and sample size**\n\nThere were 14 participants who completed the full experiment, and we trained 15 RNNs in each condition.\nWe conducted power analysis based on effect sizes reported in a prior study (Exp. 1 in Schaffner et al., 2023, which investigated the effect of reward on orientation perception). Specifically, we used the reported coefficient (Estimate = −0.16, SE = 0.09) from the Bayesian hierarchical linear regression model to approximate a standardized effect size (Cohen’s f ≈ 2.0). The effect reported estimates how reward influence the performance in oblique orientations. Since our design includes 5 orientation bins, we applied a Bonferroni correction for multiple comparisons, resulting in a corrected α = 0.01. Power analysis revealed that with 14 participants, our study already achieves well above 90% power to detect an effect of reward in the oblique orientation.\n\n**2. Include a discussion of the bias predicted by the normative model.**\n\nThank you for the insightful comment. Indeed, our analysis, theoretical framework, and RNNs focused on changes in variability over time, rather than jointly explaining bias and variability. While we briefly acknowledged this in the discussion, we agree it deserved expansions in discussion. \n\nEfficient coding theory has successfully jointly explained perceptual bias and discrimination thresholds (Wei & Stocker, 2015, 2017). A direct extension of this framework to memory would predict that **bias and variability remain coupled across delays**. However, our results diverge from this: while variability increases substantially over longer delays, bias remains relatively stable. At short delays, the observed bias and variability align with prior empirical findings and theoretical predictions, suggesting that performance reflects established principles of perceptual encoding. The **temporal dissociation between bias and variability** highlights a distinction between encoding and memory maintenance—a key contribution of our study. \n\nWe see two possible explanations: \n- (1) Recent theoretical work (e.g., Hahn & Wei, 2024) showed that bias reflects the combination of two opposing forces: a repulsive component from encoding (asymmetric likelihood function) and an attractive component from the prior used during Bayesian decoding. These effects can co-exist and vary independently. If, across delay or conditions, repulsion increases while prior-driven attraction also grows, their net influence may cancel, resulting in stable observed bias. \n- (2) WM maintenance engages neural populations distinct from those involved in initial sensory encoding. While encoding likely relies on sensory areas with fixed tuning curves that tile the feature space—as assumed in efficient coding models—WM relies on distributed activity in prefrontal and parietal cortices, where neurons show mixed and flexible selectivity. These dynamics may not preserve the original encoding structure, allowing variability to increase over time without corresponding changes in bias. This pattern aligns with our RNN simulations, which better capture memory-related dynamics than static sensory encoding.\n\nBased on another reviewer’s comment, we investigated more systematically, regarding how noise levels affect memory in RNNs. When noise levels are the same between training and testing, we found that bias is relatively stable compared to variability (consistent with our data). Interestingly when varying noise from training and testing we could observe stronger effect of delay on the bias (see response to the point 3)\n\n**3. Results from larger networks, different noise levels, or alternative loss weightings.**\n\n**Larger network.** We trained larger RNNs with 128 recurrent units (vs. 32 in the main paper). The results of variability (Fig. 5) are similar under a larger network, while the absolute variance was lower, suggesting improved precision with larger networks. Interestingly, bias patterns diverged: larger RNNs showed stronger biases over time—specifically, increased attraction to the prior in the baseline context, and to reward in the reward context. These patterns are not consistent with the repulsion we and others found for orientations in WM.\n\n**Noise levels.** For the smaller RNN originally reported in our manuscript, we systematically investigated RNNs trained and tested under different levels of internal sensory noise (σ = 0.1, 0.5, 0.8 of the recurrent weight). Our key findings are:\n- Matched Training and Testing Noise:\n  - The reported variability pattern (Fig. 5) holds consistent across different levels of noise.\n  - For bias, there is a repulsion to the prior in low noise (σ = 0.1) but transitioned into attraction when noise increased to σ = 0.8. \n- Testing at *lower* noise than training (e.g. trained on σ = 0.5, tested on σ = 0.1). \n  - The reported variability pattern (Fig. 5) holds consistent. \n  - There were attractive biases toward the prior or reward dominate, when tested with lower noise.\n- Testing at *higher* noise than training (e.g. trained on σ = 0.1, tested on σ = 0.5) \n  - The variability results we reported in the paper broke down. \n  - Bias became more repulsive from prior or high-reward stimuli, consistent with larger noise amplifying repulsive bias.\n\n**Alternative loss weightings.** we trained additional RNNs with zero L2 penalty on recurrent weights and activations, using zero initialization to prevent overfitting. We observed that, compared to the regularized models (1e-4 L2 penalty):\n- These models exhibited behavior holds consistent, whit variability affected by both prior and reward. \n- They developed similar recurrent weights (local excitation, broad inhibition) consistent with ring-attractor dynamics.\n- These results suggest that resource-efficient behavior can emerge from task optimization alone, even in the absence of hard-coded limitations. Thus, “limited resources” in working memory may reflect rational strategy to task goals rather than strict architectural bottlenecks.\n\n**Conclusions.** Across these settings, our core findings—variance modulation by prior and reward, and the emergence of attractor-like dynamics—remain qualitatively stable. Including these results in the revision will significantly improve the manuscript.\n\n**4. Learning results across 100 trials.**\n\nWe agree that quantifying learning trajectories would provide a more rigorous picture of reward learning. In both the reward and baseline context, the participants completed 5 runs of learning, with 20 trials per run 100 trials in total. In the reward context, the percentage of trial participants choosing the diagonal (highest-reward) stimuli is already at 56% in the first run (chance level = 25%). This percentage increased over time and plateaued in the third run close to 70% and remained at the same level throughout the training. When binning the trials into smaller bins (5 trials per bin), we found that learning reached an asymptote (70% choosing the highest-reward stimuli) at around 60 trials. These patterns were consistent across almost all participants. \n\nIn the baseline context, the choices for the diagonal stimuli (now not high-reward) of all participants reaches the chance level 25% around trial 65 which is at run 4, with all the participants near 25% in the final block. In the revised manuscript, we will add figures to illustrate these learning curves. \n\n**5. Results from frozen recurrent weights.**\n\nThanks for this excellent and important question! We performed two sets of transfer learning experiments:\n- **Transfer across prior distributions (uniform → natural).** We first trained the 15 RNNs under a uniform distribution + baseline context. After training, we froze the recurrent weights and **re-trained only the readout weights** in the natural distribution + baseline context. We found that:\n  - The variability pattern remained similar to the original uniform + baseline models: variability was uniform across orientations and did not reflect the natural distribution prior.\n  - In contrast, the bias showed a shift toward the natural distribution + baseline pattern.\n- **Transfer across contexts (baseline → reward).** We trained 15 RNNs in the uniform distribution + baseline context, then froze the recurrent weights and retrained the readout in the uniform distribution + reward context. We found:\n  - For variability, there was a reduction at high-reward orientations, but this effect was weaker than in models fully trained in the reward context. Hence, both recurrent and readout weights contribute to reward-driven variability modulation.\n  - For bias, the transferred models closely matched the fully trained reward context models.\n\nTherefore, the effect of prior or reward policy on memory **variability is rooted in the changes of recurrent dynamics**, while **the bias can be more flexibly shaped by the readout layer based on prior and reward**. \n\n**Training algorithm.** We used BPTT and SGD for training, but we do not think that our results hinge on the specific training algorithm. We recognize that developing biologically plausible alternatives to BPTT remains an active area of research, and several notable approaches have demonstrated similar outcomes (Miconi, 2017, Murray, 2019, and Bellec et al., 2020). Nonetheless, BPTT and SGD remain the most widely used training methods for RNNs today. We will acknowledge this point in the revised Discussion section."}, {"review_id": "EPMfWS07KI", "reviewer": "Reviewer_Tu7B", "summary": "The paper examines the question on the basis of what humans can effectively allocate working memory resources - utility or environmental statistics. Experiments with real participants and model experiments with RNN were conducted.", "questions": "1. How many people took part in the experiment?\n2. How correct is it to use RNN as a memory model?\n3. Are there other approaches to assessing the allocation of working memory resources?\n4. It is useful to conduct experiments with a very simple model, MLP, to demonstrate that the memory in the RNN is actually used and the model does not simply remember the pattern present in the data.\n\nAt the moment my rating is “Borderline reject”, but I will consider increasing the rating if the answers to the questions are convincing.", "limitations": "The limitations of research are discussed sufficiently in the Discussion section, but I recommend that authors describe limitations into a separate section to improve the perception of the work.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. A comparison was made with the results of real participants.\n\n**Weaknesses:**\n1. A very simple model, RNN, is used for comparison. Its use for assessing the mechanisms of human memory raises questions.\n\n**Quality**\n\nThe experiments sufficiently confirm the authors' сlaims on the issues discussed in the paper.\n\n**Clarity**\n\nThe paper is well written and easy to read.\n\n**Significance**\n\nAs far as I can judge from my limited expertise, the research presented in the paper is important.\n \n**Originality**\n\nThe paper lacks a Related Work section and a brief survey of relevant work given in the Introduction section. As far as I can tell, the work is quite original, but I recommend that the authors expand the comparison section and put it into a separate section.\n\nI should note that I am not an expert in neuroscience and cognitive science and may be missing important results for these areas and incorrectly evaluate the significance and originality of the work.", "was_revised": true, "final_justification": "The authors provided sufficient clarifications to my questions, which allows me to increase my score. However, since taking these answers into account in the final version of the paper requires its significant revision, which cannot be assessed, I will increase the score to 4.", "rebuttal": "We thank the reviewers for their constructive comments, and for recognizing that the research topic we address is important. Based on the reviewer’s comments, in the revision, we will expand the Introduction to justify how RNNs are widely used in neuroscience literature to study working memory, and we will add a dedicated Related Work section and Limitation section to improve the clarity. In addition, we showed that RNNs can generalize to untrained longer delays and untrained stimuli. Below we addressed each comment by the reviewer.\n\n**1. Validate the use of RNN for assessing the mechanisms of human memory.**\n\nWe agree that RNNs are simplifications of biological memory systems. Our goal is not to claim that RNNs fully capture the mechanisms of working memory, but to use them as functional models to explore how reward and prior information shape memory dynamics.\n\nRNNs are widely used in neuroscience to model working memory due to their ability to maintain information over time and generate dynamics resembling cortical activity observed in humans and non-human primates (e.g., Compte et al., 2000; Wang, 1999; Bouchacourt & Buschman, 2019; Wimmer et al., 2014; Esnaola-Acebes et al., 2022; Yang & Wang, 2020; Wang, 2021). When trained on task-relevant objectives, they often develop ring-attractor-like dynamics, thought to underlie memory for simple visual features  in the brain.\n\nWhile RNNs have been extensively used to study working memory, their adaptation to varying reward structures and stimulus statistics remains underexplored. We deliberately used a minimal architecture to isolate the influence of these factors. Despite its simplicity, the model reproduced key behavioral patterns observed in human data. Overall, we used RNN as the model system for working memory based on the rich literature on this topic. In the revision, we will expand the background on how RNNs are used to study memory in neuroscience and clarify our rationale for using minimal models to study the influence of training context.\n\n**References**\n\nWimmer, K., Nykamp, D. Q., Constantinidis, C., & Compte, A. (2014). Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nature neuroscience, 17(3), 431-439. doi: 10.1038/nn.3645.\n\nCompte, A., Brunel, N., Goldman-Rakic, P. S., & Wang, X. J. (2000). Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. Cerebral cortex, 10(9), 910-923. doi: 10.1093/cercor/10.9.910.\n\nBouchacourt, F., & Buschman, T. J. (2019). A flexible model of working memory. Neuron, 103(1), 147-160. doi: 10.1016/j.neuron.2019.04.020\n\nWang, X. J. (2021). 50 years of mnemonic persistent activity: quo vadis?. Trends in Neurosciences, 44(11), 888-902.\n\nYang, G. R., & Wang, X. J. (2020). Artificial neural networks for neuroscientists: a primer. Neuron, 107(6), 1048-1070.\nEsnaola-Acebes, J. M., Roxin, A., & Wimmer, K. (2022). Flexible integration of continuous sensory evidence in perceptual estimation tasks. Proceedings of the National Academy of Sciences, 119(45), e2214441119.\n\n**2. Add Related Work section in Introduction**\n\nTo clarify, in the original manuscript, we discuss **relevant work** in the Introduction and Section 3. These include:\n- Foundational findings on working memory (WM) capacity and resource limit (Luck & Vogel, 1997, 2013; Ma et al., 2014; Pertzov et al., 2017),\n- Evidence for flexible allocation of WM resource based on attention (Zhang & Luck, 2008; Yoo et al., 2018),\n- Contrasting non-effect evidence on reward-modulation of WM resource (Brissenden et al., 2023; Van den Berg et al., 2023),\n- Neural and computational models of WM using RNNs and attractor dynamics (Compte et al., 2000; Wimmer et al., 2014; Bouchacourt & Buschman, 2019), and\n- Normative models of efficient coding and utility optimization under constraints (Wei & Stocker, 2015; Morais & Pillow, 2018; Hahn & Wei, 2024).\n\n**Research gap:** While these studies have advanced our understanding of WM and its limitations, it remains unclear whether and how WM resources can be allocated efficiently to maximize utility. This gap spans both behavioral and modeling perspectives. Our work fills this gap by:\n- Providing behavioral evidence that WM resources are adaptively allocated to reward-maximizing features after value learning,\n- Proposing a framework that explains how prior and reward jointly influence WM resource allocation, and extending the model to incorporate temporal dynamics,\n- Using simple but interpretable RNN models to show that such adaptive allocation can emerge through training objectives and stimulus distributions.\n\nBased on this and the previous comment from the reviewer, In the revision, we will expand and consolidate a dedicated Related Work section to highlight the previous studies, especially the neuroscience studies that used RNN to model working memory and the research on the effect of reward on working memory, and the research gaps to better explain our contributions.\n\n**3. Sample size.**\n\nThere were 14 participants who completed the full experiment (14 subjects X 2 sessions X 90 minutes per session), and we trained 15 RNNs in each context and prior distribution conditions. We also conducted a power analysis regarding the number of subjects. The result showed that the 14 subjects achieved well above 90% power to detect an effect of reward in the oblique orientation. See response point 1 to reviewer Y8u3. \n\n**4. Other assessment of the allocation of working memory resources.**\n\nYes, there are several well-established approaches to studying working memory resources in cognitive science. \n- One line of research investigated the format and the capacity of working memory by studying how memory precision declines when subjects were asked to remember more items (e.g., Cowan, 2002; Zhang and Luck, 2008; van den Berg et al., 2012). \n- The other line of study focused on the effect of attentional cue on working memory. They involved asking subjects to remember multiple items in a trial and at the same time presenting an attentional cue to inform the subjects which item is more important (more likely to be probed/asked at the end). The general findings are that people remembered the cued item better than the un-cued items. Note that this kind of attentional cueing effects are well-established (cited studies in the introduction: Zhang and Luck, 2008, Dube et al., 2017, Yoo et al., 2018) and are not the research interest in our study. Here we aim to isolate the effect of reward by implementing a working memory task with a single item per trial. \n\nBesides the manipulation of reward in the behavioral experiment, we also implemented a L2 regularization on total neural activity in the RNN, which can be considered as a form of resource constraints (see more in the response to Reviewer jY8u3’s point 3). \n\n**References**\n\nN Cowan, The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behav Brain Sci 24, 87–114, discussion 114–185. (2001).\n\n Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453(7192), 233-235.\nVan den Berg, R., Shin, H., Chou, W. C., George, R., & Ma, W. J. (2012). Variability in encoding precision accounts for visual short-term memory limitations. Proceedings of the National Academy of Sciences, 109(22), 8780-8785.\n\n**5. RNNs do not simply remember the input-output patterns.**\n\nThanks for raising an important point about ensuring that the RNN uses its memory dynamics rather than memorizing input-output patterns. To address this, we evaluated the memory in (1) **longer delays** than it was trained on and (2) **unseen stimuli**. \n- Even at delays of 6–8s (trained with ~4s), the RNN maintained the memory well, with a low memory variability (<8°). We also observed robust effects of prior and reward on memory variability, consistent with adaptive use of memory over time. \n- We trained and tested RNNs in different levels of recurrent weight noise levels (σ = 0.1, 0.5, 0.8), and observed the reported variability pattern (Fig. 5) holds consistent when the models are tested in lower noise than they are trained. Further details are discussed in response point 3 to the reviewer Y8u3.\n\nPlease also see our responses point 5 to reviewer Y8u3, where we now show that the **RNN recurrent weights are needed to explain the data**. Changing the readout weights alone can not reproduce how memory variability changes with time, prior and reward policy.\n\nWe agree that simpler models like MLPs can be useful baselines. We trained an MLP on the same task and found that it could accurately classify orientations when evaluated immediately after stimulus offset. Previous studies also found that DCNN and MLP trained on natural images showed higher sensitivity to cardinal orientations which are more prevalent in natural environments, similar to humans and our results here (the cited study in our manuscript: Benjamin et al., 2022). However, these models are purely feedforward without temporal dynamics, and therefore are not well-suited for modeling memory dynamics, which is the focus here. In contrast, RNNs have a dynamic internal state that evolves without additional input. \n\nThat said, if the reviewer has a specific use in mind for how an MLP could help elucidate the issue further, we would be very happy to add that analysis in future work.\n\n**References**\n\nBenjamin, A. S., Zhang, L. Q., Qiu, C., Stocker, A. A., & Kording, K. P. (2022). Efficient neural codes naturally emerge through gradient descent learning. Nature Communications, 13(1), 7972.\n\n**6. Describe limitations into a separate section**\n\nThank you for the helpful suggestion. We agree that clearly stating the limitations in a dedicated section can improve clarity and transparency and we will do so in the revised manuscript."}, {"review_id": "Oj7mNlds60", "reviewer": "Reviewer_jK1W", "summary": "The manuscript investigates how precision in human working memory (WM) is allocated by both environmental statistics and learned reward contingencies with limited memory resources. The authors derive an efficient-coding rule in which Fisher information is proportional to the product of the stimulus distribution and a value term, and they show that a recurrent neural network trained under corresponding loss functions reproduces the human patterns. Together, these results support that WM precision is dynamically steered toward stimuli with higher prior and values.", "questions": "1.\tCould you report formal comparisons of variability parameters between baseline and reward contexts?\n\n2.\tPlease define all symbols in formulas.\n\n3.\tHave you trained RNNs without the activity regularized, or with a weaker coefficient? If the reward-dependent allocation persists, that would suggest that resource constraint is not an explanation.\n\n4.\tCan you justify the power of sample size?", "limitations": "No, there’s no explicit limitations mentioned in the discussion.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n•\tThe study combines behavioral data, a normative model, and RNN simulations, providing a multi-angle examination of the hypothesis. The motivation is clear.\n\nWeaknesses\n\n•\tIn Fig. 2 the difference between baseline and reward contexts is not obvious by eye; no statistical comparison of fitted key parameters (e.g., A,B, phi,n) is reported.\n\n•\tSymbols (e.g., epsilon, kappa, I1, I0) between line 99 and 104, in Eq. 1 and  (line 142) v(theta) are never defined, which hampers reproducibility.\n\n•\tLine 155 claims that the J_opt depends on t; but the explicit formula is unclear.\n\n•\tThe RNN uses an L2 activity penalty. It is not shown whether removing or relaxing this penalty changes the results, leaving open whether “resource limits” drive the effect.\n\n•\tFourteen participants limit generalizability.", "was_revised": true, "final_justification": "My concerns are adequately addressed.", "rebuttal": "We thank the reviewers for their constructive comments, and especially for recognizing our approach that integrates new behavioral data, cognitive modeling, and neural network models, providing multi-angle examinations on the research topic. To address the issues raised, in the revision, we will describe more statistical tests and include additional visualization to clarify the effect or reward in behavioral results. We will also add explanations on symbols in equations, which were missing in the original manuscript. Below we addressed each comment by the reviewer.\n\n**1. Formal comparisons of variability between baseline and reward contexts**\n\nFor better visualization and statistical analysis of the **behavioral results** in Fig. 2, we will do the following: \n\n- (1) add a figure which doesn’t use orientation as the x-axis, but uses the “distance to the cardinal orientations” as the axis (bin1 to bin5, from the most cardinal to the most diagonal orientations; this involves folding the original x-axis in Fig. 2 twice). The bonferroni corrected paired t-test on this type of data binning showed that:\n  - For baseline context, the variability in delay=5s is significantly higher compared to the delay=1s in the bin4 (relatively close to diagonal, *p* < .01).\n  - For reward context, the variability in delay=5s is significantly higher compared to the delay=1s in the bin2 (relatively close to the cardinal, *p* < .05)\n- (2) add figures which change the way we group the results: in each delay condition, we will plot the variability and bias in reward and baseline context to make the comparison more obvious. Since there was a significant 3-way interaction in variability indicated by a linear mixed-effects model (context x delay x orientation, *p* < .05), we also provide the statistical results grouped by the delays. \n  - For delay=1s, there is no significant 2-way interaction (context x orientation, *p* = .161), indicating the pattern of the variability over orientation remain similar across the contexts; \n  - For delay=5s, there is a significant 2-way interaction (context x orientation, *p* < .05), showing that the pattern of the variability over orientation is significantly different across two contexts. \n\nAdditionally, to address this question, For the **modeled resource distribution** in Fig. 3, we will also add a new figure that groups the resource distribution by delay in the revised version. Based on the cluster-based permutation test, there is no significant different orientation cluster for resource across contexts in delay=1s, while two diagonal clusters were identified for delay=5s where there are more resource allocated to the diagonal (high-reward) stimuli in the reward context compared to the baseline (*p* < .001).\n\n**2. Statistical comparison of fitted parameters.**\n\nFor the formula $\\bar{J}(\\theta) = A \\cdot \\left(1 - B \\cdot |\\sin(2\\theta + \\phi)|\\right)^n$, Fig.3 shows that resource distributions exhibited **a stronger oblique effect in long delay in the baseline context** (the “W” pattern in Fig.3A), but **a weaker oblique effect in long delay in the reward context** (a shallower “W” in Fig 3B). These effects are **reflected in the parameter $B$**. Indeed, we found a significant interaction effect (context x delay, *p* < .05) when testing the best-fitted $B$ values. We will report the statistical tests on all the fitted parameters in the revision. We describe them in more details here:\n- For $A$: the linear mixed-effects model showed no significant interaction effect (context x delay, *p* > .05), and no significant main effect (*p* > .5). The bonferroni corrected one-sample t-test showed that A is significantly larger than 0 (*p* < .001) in all contexts and delays.\n- For $B$: the linear mixed-effects model showed a significant interaction effect (context x delay, *p* < .05), and a marginal significant effect for delay (*p* = .06). The bonferroni corrected one-sample t-test showed that B is significantly larger than 0 (*p* < .001) in all contexts and delays.\n- For $\\phi$: the linear mixed-effects model showed a marginal interaction effect (context x delay, *p* = .06), and no significant main effect (*p* > .05). The bonferroni corrected one-sample t-test showed that $\\phi$ is only significantly larger than 0 in delay=5s in both context, while not significantly different from 0 in delay=1s in both context.\n- For $n$: the linear mixed-effects model showed no significant interaction effect (context x delay, *p* > .05), but a significant effect for delay (*p* < .05). The bonferroni corrected one-sample t-test showed that n is significantly larger than 0 (*p* < .001) in all contexts and delays.\n\n**3. Activity regularization and working memory resources.**\n\nIn addition to the models reported in the main paper, we also trained RNNs without weight or activation regularization while keeping all the other settings the same. We found similar behavioral patterns even without the regularization. This suggested that resource-efficient behavior can emerge from task optimization alone, even in the absence of hard-coded limitations. See details in response point 3 to reviewer Y8u3 regarding \"Alternative loss weightings\".\n\n**4. Sample size and generalizability.**\n\nAt this stage, we are not able to provide additional data or replication, but we did conduct a power analysis regarding the number of subjects. The result showed that the 14 subjects achieved well above 90% power to detect an effect of reward in the oblique orientation. See response point 1 to reviewer Y8u3. Additionally, our study uses a tightly controlled cognitive task with a within-subject design that is typical in cognitive experiments.\n\n**5. Define all symbols in formulas.**\n\nThank you for pointing out the missing symbol definitions. We now explicitly define all mathematical symbols upon their first appearance for clarity and reproducibility, and they will be added to the revised manuscript near their respective equations.\n- Specifically on **Page 4**:\n  - For von Mises distribution equation, $\\epsilon$ denotes the angular memory error, and $\\kappa$ is the concentration parameter.\n  - $I_0(\\kappa)$ and $I_1(\\kappa)$ are the modified Bessel functions of the first kind of order 0 and 1, respectively.\n  - $J \\sim \\mathrm{Gamma}(\\bar{J}/\\tau, \\tau)$ specifies that memory precision varies across trials, drawn from a Gamma distribution with shape parameter $\\bar{J}/\\tau$ and scale parameter $\\tau$. \n    - $\\bar{J}$ is the mean of the resource parameter $J$. \n    - $\\tau$ is the scale parameter of the Gamma distribution, controlling variability in $J$ across trials.\n- On **page 5**:\n  - $v(\\theta)$ in line 142 is a typo which should be context-dependent rewards $r(\\theta)$, and $\\theta$ is the orientation.\n  - $C$ in the equation $\\int J(\\theta)^{\\beta} \\ d\\theta < C $ represents total resource constraint, and $\\beta$  is an exponent term used in the power-law efficient code (Morais and Pillow, 2018)\n  - $p$ from $L = \\int p(\\theta)\\ J(\\theta)^{-\\frac{p}{2}}\\ d\\theta $, and $q$ from $ J_{\\text{opt}} \\propto p(\\theta)^q \\$ are also exponent terms used in the power-law efficient code (Morais and Pillow, 2018)\n\n\n**6. Clarify Line 155 $J_{opt}$ depends on $t$.** \n\nIn section 3.3 we introduced $J(\\theta, t)$, which is the allocation of resource $J$ that is a function of orientation $\\theta$ and time $t$. It is a direct extension of the equation $L = \\int p(\\theta)\\ J(\\theta)^{-\\frac{p}{2}}\\ d\\theta $ at line 128, but here we introduce a time domain, allowing the resource $J$ to change with time. \n\nThe results stated in Line 155 $J_{opt}(\\theta,t) \\propto p(\\theta)^q$ is the outcomes of optimization (minimizing the loss) in Line 154. We found that even after considering the time domain, the $J_{opt}$ only depends on orientation. That is, the same strategy of resource allocation (more resource to more frequent stimuli) is the same at any time points throughout the memory delay. As a consequence, the difference in resources allocated between frequent (cardinal orientations) and infrequent (diagonal orientations) stimuli will accumulate over time, leading to a stronger oblique effect with longer delay (consistent with our data Fig. 2A)."}, {"review_id": "WfOIXew3tq", "reviewer": "Reviewer_QE77", "summary": "The authors study human behavior from a computational perspective in a delayed-estimation WM task. More specifically, they investigate how learned reward structures effect WM. No structure in learned rewards led to a sensitivity to naturalistic (in this case cardinal orientations) priors. This effect, however, could be overwritten based on learned rewards and was modulated by the delay of the estimation. This was then captured by ideal observer and recurrent neural network models.", "questions": "1. The authors write that “We decomposed memory errors into bias and bias-corrected (with a polynomial regression) variability.” It is not clear to me at all what was done here, but this seems particularly important as it is the foundation for the main analysis. A better explanation is desperately needed. I was also wondering why the composition was done in the first place. Why not look just at MSE or similar? That seems much simpler. \n2. Do the authors think that, if the reward structure was communicated directly instead of learned, the effect would disappear? The discussion seems to hint at this at least. If so, where is this accounted for in the computational theory?\n3. My guess is that the RNN is only modeling the WM component of the task, is that correct? Perhaps be good to mention that somewhere.\n4. For the RNN models, the fact that they were trained with gradient descent does not really seem relevant. Wouldn’t you have gotten the same behavioral characteristics with any reasonable learning algorithm?", "limitations": "Discussion was sufficient.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Introduction set up the work nicely and placed it into context.\n2. The description of the behavioral task was easy to follow.\n3. Figuring out how learned rewards interact with working memory is an important research direction and the results presented in Figure 3 were quite clear and show an interesting effect.\n\nWeaknesses:\n1. Looking at Figure 2, the only result that seemed clear to me was the oblique effect in Figure 2A. For instance, the authors write that, for the reward context, “diagonal orientations (now associated with higher reward) were maintained well while cardinal orientations (now associated with lower reward) had their memory precision dropped.” But that is not apparent to me looking at either Figure 2B or 2D.\n2. I had a hard time following much of the modeling (even though I consider myself an expert in both ideal observer models and recurrent neural networks). In the present state, I don’t think it is ready for publication. Just to give some pointers:\n* I_0 and I_1 on p4 were never introduced, making it impossible to understand what J is.\n* The same equation is shown twice on p4.\n* “We constructed the resource allocation as a function of orientations by fitting the model to individual data in different contexts and delays.” How this is done is not described at all.\n* The first two equations on p5 contain an expectation but this is dropped for the third equation. The reason for this is unclear to me. There also seems to be a minus missing in the third equation.\n* v(\\theta) is never introduced. Is it the same as r(\\theta)?\n* J(\\theta, t) is introduced but an expression is never given.", "was_revised": true, "final_justification": "The authors' have addressed some of my concerns in the rebuttal, leading me to increase my score to 3. However, to really judge with confidence how the proposed changes would effect the readabiltiy of the paper, I would have to see how they are integrated in the text (which I don't think is easy to realize in this rebuttal-style format that doesn't allow uploading an updated PDF).", "rebuttal": "We thank the reviewers for their constructive comments. We appreciate the reviewer’s positive comments highlighting the clear motivation and contextual framing, the well-explained behavioral task, and the importance and clarity of the work. \nTo address issues raised, in the revision, we will include additional visualizations to clarify the results on reward context and the memory delay. We will add detailed explanations that are needed for the equations, symbols and how we decompose the bias and variability. Below we addressed each comment by the reviewer.\n\n**1. Clarify the behavioral effects in Fig. 2.**\n\nTo address this point, we will add new figures and include more details of statistical results in the revision. Please see more details in our response point 1 to reviewer jK1W.\n\n**2. Clarify the modeling part.**\n\nThank you for your helpful feedback regarding clarity in the modeling section. We apologize for the confusion and have taken steps to improve the manuscript accordingly.\n- **Define symbols.** \n  - We now explicitly define all mathematical symbols upon their first appearance to enhance readability and reproducibility, please see response point 5 to reviewer jK1W regarding the symbols. \n  - In addition, $J(\\theta, t)$ in Sec 3.3 is the allocation of resource $J$ that is a function of orientation $\\theta$ and time $t$. It is a direct extension of the equation $L = \\int p(\\theta)\\ J(\\theta)^{-\\frac{p}{2}}\\ d\\theta $ at line 128, but here we introduce a time domain, allowing the resource $J$ to change with time. At Line 155, we found that even after considering the time domain, the $J_{opt}$ only depends on orientation. That is, the same strategy of resource allocation (more resource to more frequent stimuli) is the same across time points throughout memory delay. As a consequence, the difference in resources allocated between frequent (cardinal orientations) and infrequent (diagonal orientations) stimuli will accumulate over time, leading to a stronger oblique effect with longer delay, consistent with our observations in Fig. 2A.\n- **Repeated equation.** We will also correct the redundancy in the text where the same equation appeared twice on page 4 in the revised version. \n- **Clarify resource model fitting.** To break down the sentence “We constructed the resource allocation as a function of orientations by fitting the model to individual data in different contexts and delays.”:\n  - The memory error in each context and delay is separately fitted to the model\n  - We fitted a function $\\bar{J}(\\theta) = A \\cdot \\left(1 - B \\cdot |\\sin(2\\theta + \\phi)|\\right)^n$ where the mean resource $\\bar{J}$ is allowed to vary as a function of orientation $\\theta$. The free parameters in the functions $A$, $B$, $\\phi$ and $n$ are used so the function has the potential to flexibly form shapes such as a flat line or the “W” shape we found in the paper. We fitted this function using the memory errors (residuals) for each context and each delay separately. \n  - After fitting these free parameters to the memory error of each subject, context and delay, we visualize the distribution of resource $\\bar{J}(\\theta)$, by plotting the best-fit functions and averaged across subjects (Fig. 3)\n\n- **Correct the equation.** You are absolutely right that the expectation symbol was inadvertently dropped in the third equation. \n  - We will correct it in the revised manuscript to ensure consistency and mathematical accuracy: $ L = \\int p(\\theta)\\ r(\\theta)\\ \\mathbb{E}|\\hat{\\theta} - \\theta| \\ d\\theta$. \n  - Note that the objective changes from maximizing the second equation (the reward) in Page 5 to minimizing the loss so there is no minus sign. To make it clearer, we change the left side of the third equation from $R$ (reward) to $L$ (loss). \n\n**3. Validate the bias-variability decomposition.**\n\nThe memory error could come from both bias and variability as shown in Fig. 1G. For each context-delay pair, we performed a polynomial regression for the behavioral error as a function of stimulus orientation. \n- Specifically, we fit a low-degree polynomial (degree 3) regression to estimate the expected bias as a function of orientation. \n- The fitted bias $\\hat{b}(\\theta)$ was then subtracted from the raw errors to obtain bias-corrected errors (residuals) for subsequent variability analyses. \n- The variability (Fig. 2A and 2B) was computed as the standard deviations of these residuals. \n- The bias was computed by simply averaging the raw (signed) errors within each bins (Fig. 2C and 2D). \n\nIn the revision, we will include a figure showing the bias and the fitted bias (from the polynomial regression).\n\nAs we explained in Fig. 1G, estimation errors have two different components—variability and bias, and in the literature researchers have observed distinct patterns between them, especially for orientation estimation. For example, Girshick et al., 2011 separately reported variability and bias, and showed that the variability patterns can be linked to environmental prior. More recent work on efficient coding theory has attempted to explain the sign and magnitude of bias and variability (Wei & Stocker, 2015, 2017). Recent studies on human performance in orientation estimation also used the same approach to decompose variability and bias (Van Bergen et al., 2015). We believe that decomposing them is the most thorough way to look at the memory dynamics on orientations. \n\n**References**\n\nVan Bergen, R. S., Ji Ma, W., Pratte, M. S., & Jehee, J. F. (2015). Sensory uncertainty decoded from visual cortex predicts behavior. Nature neuroscience, 18(12), 1728-1730.\n\nGeurts, L. S., Cooke, J. R., van Bergen, R. S., & Jehee, J. F. (2022). Subjective confidence reflects representation of Bayesian probability in cortex. Nature human behaviour, 6(2), 294-305.\n\n**4. Direct instruction versus self-learned reward structure.**\n\nIndeed, we intentionally designed the reward manipulation to be learned through experience rather than directly instructed, though direct instruction will save much effort.\n\nOur rationale was grounded in both theoretical and empirical considerations. First, there is evidence from the literature that **learned reward associations—especially via reinforcement—can produce stronger and more flexible modulations of behavior** compared to explicit instructions (e.g., Collins & Frank, 2013; Ballard & McClure, 2019). Learning the reward contingencies engages brain systems involved in reinforcement learning and may promote **deeper integration of value information** into cognitive control and working memory mechanisms, potentially enhancing generalization or transfer.\n\nSecond, in prior studies where reward structures were **explicitly cued or instructed**, effects on working memory resource allocation have often been absent (Brissenden et al., 2023, and Van den Berg et al., 2023). We hypothesized that **allowing participants to learn the value structure through trial and error would produce stronger reward-dependent modulation of memory**, which aligns with the robust effects we observed.\n\nWe agree this distinction between learned vs. instructed value could have theoretical importance, and we plan to explore this manipulation more directly in future work.\n\n**5. RNN is only modeling the WM component of the task.**\n\n**Yes, that is correct.** The RNN was designed to model the working memory (WM) component of the task, rather than the initial learning of the reward-orientation mapping in the value-learning task. The RNN model itself was trained to optimize performance in the WM task, capturing the influence of the learned value structure through reward-weighted loss during training, without explicitly modeling the value learning task in the behavioral experiment. We will clarify this point more clearly in the revised manuscript.\n\n**6. Training algorithm of the RNN**\n\nWe used backpropagation through time (BPTT) and stochastic gradient descent (SGD) for training, but we do not think that our results hinge on the specific training algorithm. We recognize that developing biologically plausible alternatives to BPTT remains an active area of research, and several notable approaches have demonstrated similar outcomes (e.g., Miconi, 2017; Murray, 2019; Bellec et al., 2020). We will revise the Discussion to acknowledge this point and clarify that our findings likely reflect properties of the trained model architecture and task objective, rather than being specific to the training algorithm used.\n\n**References**\n\nMiconi, T. (2017). Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife, 6, e20899.\n\nMurray, J. M. (2019). Local online learning in recurrent networks with random feedback. Elife, 8, e43299.\n\nBellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1), 3625."}]

Author rebuttal:

Meta-review: (a) Scientific Claims and Findings:
The paper presents a study combining behavioral data, a normative model, and RNN simulations to examine the interaction between RL and WM. Several key patterns of Human behaviours are reproduced by RNN models. The overall motivation is clear, and the authors provide useful clarifications, but some details still need attention for full clarity and reproducibility.

(b) Strengths of the Paper:

- The approach combines multiple angles, including behavioral data, modeling, and simulations, which strengthens the argument.
- The motivation behind the study is solid and well-articulated.

(c) Weaknesses and Missing Aspects:

- In some figures (like Fig. 2), differences between baseline and reward contexts aren't visually obvious, and statistical comparisons are missing.
- Some key symbols (like epsilon, kappa, etc.) aren’t defined, making it harder for others to reproduce the work.
- The explicit formula for J_opt is unclear, and the paper doesn't show whether relaxing the L2 penalty in the RNN would change the results.
- The sample size (14 participants) is small, which limits generalizability.

(d) Rebuttal Discussion and Changes:
The authors addressed most of the concerns raised, which led to a positive shift in scores. However, due to the rebuttal format, it's hard to assess how the revisions would impact the paper's readability and presentation fully. A more concrete update would help reviewers evaluate the changes better.

(e) Reasons for Acceptance/Rejection:

I recommand **accepting** this paper. The paper addresses an interesting and well-motivated problem. The novelty lies at the integration of human psychophysical experiments and ideal observer modeling. While there are still a few issues with clarity, reproducibility, and generalizability, the authors have made significant progress in addressing reviewer concerns.
Decision: Accept (poster)