Title: Efficient Allocation of Working Memory Resource for Utility Maximization in Humans and Recurrent Neural Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: working memory

reward maximization

biologically inspired networks

efficient coding

recurrent neural networks
Abstract: Working memory (WM) supports the temporary retention of task-relevant information. It is limited in capacity and inherently noisy. The ability to flexibly allocate WM resource is a hallmark of adaptive behavior. While it is well established that WM resource can be prioritized via selective attention, whether they can be allocated based on reward incentive alone remains under debate—raising open questions about whether humans can efficiently allocate WM resource based on utility. To address this, we conducted behavioral experiments using orientations as stimuli. Participants first learned stimulus–reward associations and then performed a delayed estimate WM task. We found that WM precision, indexed by the variability of memory reports, reflected both natural stimulus priors and utility-based allocation. The effects from reward and prior on memory variability both grew over time, indicating their effects in stabilizing memory representations. In contrast, memory bias was largely unaffected by time or reward. To interpret these findings, we extended efficient coding theory by incorporating time and reformulating the objective from minimizing estimation loss to maximizing expected utility. We showed that the behavioral results were consistent with an observer that efficiently allocates WM resource over time to maximize utility. Lastly, we trained recurrent neural networks (RNNs) to perform the same WM task under a 2×2 design: prior (uniform vs. natural) × reward policy (baseline vs. reward context). Human-like behaviors emerged in RNNs: memory was more stable (lower variability) for stimuli associated with higher probability or rewards, and these effects increased over time. Transfer learning showed that recurrent dynamics were crucial for adapting to different priors and reward policies. Together, these results provide converging behavioral and computational evidence that WM resource allocation is shaped by environmental statistics and rewards, offering insight into how intelligent systems can dynamically optimize memory for utility under resource constraints.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study human behavior from a computational perspective in a delayed-estimation WM task. More specifically, they investigate how learned reward structures effect WM. No structure in learned rewards led to a sensitivity to naturalistic (in this case cardinal orientations) priors. This effect, however, could be overwritten based on learned rewards and was modulated by the delay of the estimation. This was then captured by ideal observer and recurrent neural network models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Introduction set up the work nicely and placed it into context.\n2. The description of the behavioral task was easy to follow.\n3. Figuring out how learned rewards interact with working memory is an important research direction and the results presented in Figure 3 were quite clear and show an interesting effect.\n\nWeaknesses:\n1. Looking at Figure 2, the only result that seemed clear to me was the oblique effect in Figure 2A. For instance, the authors write that, for the reward context, “diagonal orientations (now associated with higher reward) were maintained well while cardinal orientations (now associated with lower reward) had their memory precision dropped.” But that is not apparent to me looking at either Figure 2B or 2D.\n2. I had a hard time following much of the modeling (even though I consider myself an expert in both ideal observer models and recurrent neural networks). In the present state, I don’t think it is ready for publication. Just to give some pointers:\n* I_0 and I_1 on p4 were never introduced, making it impossible to understand what J is.\n* The same equation is shown twice on p4.\n* “We constructed the resource allocation as a function of orientations by fitting the model to individual data in different contexts and delays.” How this is done is not described at all.\n* The first two equations on p5 contain an expectation but this is dropped for the third equation. The reason for this is unclear to me. There also seems to be a minus missing in the third equation.\n* v(\\theta) is never introduced. Is it the same as r(\\theta)?\n* J(\\theta, t) is introduced but an expression is never given.\n\nQUESTIONS:\n1. The authors write that “We decomposed memory errors into bias and bias-corrected (with a polynomial regression) variability.” It is not clear to me at all what was done here, but this seems particularly important as it is the foundation for the main analysis. A better explanation is desperately needed. I was also wondering why the composition was done in the first place. Why not look just at MSE or similar? That seems much simpler. \n2. Do the authors think that, if the reward structure was communicated directly instead of learned, the effect would disappear? The discussion seems to hint at this at least. If so, where is this accounted for in the computational theory?\n3. My guess is that the RNN is only modeling the WM component of the task, is that correct? Perhaps be good to mention that somewhere.\n4. For the RNN models, the fact that they were trained with gradient descent does not really seem relevant. Wouldn’t you have gotten the same behavioral characteristics with any reasonable learning algorithm?\n\nLIMITATIONS:\nDiscussion was sufficient.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript investigates how precision in human working memory (WM) is allocated by both environmental statistics and learned reward contingencies with limited memory resources. The authors derive an efficient-coding rule in which Fisher information is proportional to the product of the stimulus distribution and a value term, and they show that a recurrent neural network trained under corresponding loss functions reproduces the human patterns. Together, these results support that WM precision is dynamically steered toward stimuli with higher prior and values.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n•\tThe study combines behavioral data, a normative model, and RNN simulations, providing a multi-angle examination of the hypothesis. The motivation is clear.\n\nWeaknesses\n\n•\tIn Fig. 2 the difference between baseline and reward contexts is not obvious by eye; no statistical comparison of fitted key parameters (e.g., A,B, phi,n) is reported.\n\n•\tSymbols (e.g., epsilon, kappa, I1, I0) between line 99 and 104, in Eq. 1 and  (line 142) v(theta) are never defined, which hampers reproducibility.\n\n•\tLine 155 claims that the J_opt depends on t; but the explicit formula is unclear.\n\n•\tThe RNN uses an L2 activity penalty. It is not shown whether removing or relaxing this penalty changes the results, leaving open whether “resource limits” drive the effect.\n\n•\tFourteen participants limit generalizability.\n\nQUESTIONS:\n1.\tCould you report formal comparisons of variability parameters between baseline and reward contexts?\n\n2.\tPlease define all symbols in formulas.\n\n3.\tHave you trained RNNs without the activity regularized, or with a weaker coefficient? If the reward-dependent allocation persists, that would suggest that resource constraint is not an explanation.\n\n4.\tCan you justify the power of sample size?\n\nLIMITATIONS:\nNo, there’s no explicit limitations mentioned in the discussion.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper examines the question on the basis of what humans can effectively allocate working memory resources - utility or environmental statistics. Experiments with real participants and model experiments with RNN were conducted.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. A comparison was made with the results of real participants.\n\n**Weaknesses:**\n1. A very simple model, RNN, is used for comparison. Its use for assessing the mechanisms of human memory raises questions.\n\n**Quality**\n\nThe experiments sufficiently confirm the authors' сlaims on the issues discussed in the paper.\n\n**Clarity**\n\nThe paper is well written and easy to read.\n\n**Significance**\n\nAs far as I can judge from my limited expertise, the research presented in the paper is important.\n \n**Originality**\n\nThe paper lacks a Related Work section and a brief survey of relevant work given in the Introduction section. As far as I can tell, the work is quite original, but I recommend that the authors expand the comparison section and put it into a separate section.\n\nI should note that I am not an expert in neuroscience and cognitive science and may be missing important results for these areas and incorrectly evaluate the significance and originality of the work.\n\nQUESTIONS:\n1. How many people took part in the experiment?\n2. How correct is it to use RNN as a memory model?\n3. Are there other approaches to assessing the allocation of working memory resources?\n4. It is useful to conduct experiments with a very simple model, MLP, to demonstrate that the memory in the RNN is actually used and the model does not simply remember the pattern present in the data.\n\nAt the moment my rating is “Borderline reject”, but I will consider increasing the rating if the answers to the questions are convincing.\n\nLIMITATIONS:\nThe limitations of research are discussed sufficiently in the Discussion section, but I recommend that authors describe limitations into a separate section to improve the perception of the work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript investigates whether humans can efficiently allocate their working memory (WM) resources to maximize expected reward, and whether RNNs trained under similar constraints exhibit the same behavior. \n\nBehavioral side: 14 adult participants first learned stimulus–reward mappings and then performed an orientation-recall task after 1s or 5s delays. When rewards were uniform, memory precision followed the classical oblique effect (better for cardinals). When high rewards were assigned to oblique orientations, that pattern gradually inverted over the delay, indicating value-guided WM maintenance.\n\nComputational side: RNNs (32 units) trained with BPTT yield results that resemble those from human behavioral data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nWhether WM reflects learned value, not just attention, is still debated; the paper provides relevant behavioural evidence as well as a modeling framework.\n\nExperimental design is clever. By making reward oppose the natural orientation prior, the design neatly teases apart prior vs. value effects.\n\nCombining data, theory and modeling with RNNs helps triangulate the mechanism and increases the work’s breadth.\n\nVariability–bias decomposition, mixed-effects modelling and VP fits are appropriate and clearly reported.\n\n\nWeaknesses:\n\nSmall sample size (14 subjects) limits statistical power.\n\nReward effects appear only in variability, not bias. The discussion speculates about encoding vs. maintenance, but the normative framework itself ignores bias.\n\nOnly one tiny architecture is reported. It is unclear whether the qualitative results survive with larger networks, different noise levels, or alternative loss weightings.\n\nQUESTIONS:\nThe paper assumes participants have fully learnt the reward schedule after about 100 trials. However, the asymptotic performance is only shown descriptively. Could you provide model-based learning-curve fits and exclude incomplete learners?\n\nTraining via BPTT is hard to accomplish in a biological system. Can you obtain similar results with frozen recurrent weights or without unfolding the computational graph in time (or some other non-BPTT training)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}