Title: Estimating cognitive biases with attention-aware inverse planning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Inverse Planning

Bounded Rationality

Computational Cognitive Modeling

Inverse Reinforcement Learning

Human Interaction
Abstract: People's goal-directed behaviors are influenced by their cognitive biases, and autonomous systems that interact with people should be aware of this. For example, people's attention to objects in their environment will be biased in a way that systematically affects how they perform everyday tasks such as driving to work. Here, building on recent work in computational cognitive science, we formally articulate the \textit{attention-aware inverse planning problem}, in which the goal is to estimate a person's attentional biases from their actions. We demonstrate how attention-aware inverse planning systematically differs from standard inverse reinforcement learning and how cognitive biases can be inferred from behavior. Finally, we present an approach to attention-aware inverse planning that combines deep reinforcement learning with computational cognitive modeling. We use this approach to infer the attentional strategies of RL agents in real-life driving scenarios selected from the Waymo Open Dataset, demonstrating the scalability of estimating cognitive biases with attention-aware inverse planning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article investigates the interesting topic of how goal-directed actions are influenced by human cognitive biases. Specifically, it examines how limitations in human attention lead to biases in attending to objects, which in turn can significantly affect the actions performed.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The presentation of the main ideas is clear and consistent.\n- The main research ideas are elaborated well in the introduction and background, and further in Section 3.\n- Experiments are conducted on a synthetic dataset generated using the GPUDrive driving simulator, serving as a proof of concept.\n\n\n**Weaknesses**\n- The paper conducts only limited experiments, all performed on a synthetic dataset. While this serves as a reasonable proof of concept, it leaves unanswered how the proposed ideas would perform in real-world scenarios.\n\n- From what I understand, the experiments use only a single dataset, the Waymo Open Motion dataset, which further limits the work’s generalizability.\n\n- There are no comparisons with related works presented in the paper. The authors state that *“we build on existing work in computational cognitive science,”* but it remains unclear how the proposed model differs from prior work, both in theoretical foundations and applied aspects — for example, in relation to resource-limited planning in humans, such as the value-guided construal model.\n\n- The paper would benefit from a dedicated section for experiments and results. Currently, the presentation of experiments is scattered, making it difficult to follow. I strongly recommend adding a clear Methodology section to describe all aspects of the proposed model, followed by distinct sections for Experiments and Results.\n\n- Additionally, the manuscript lacks a Conclusion section. I suggest including one, even if brief, to summarize key findings and outline future work.\n\n- Figure 2 is difficult to interpret. There does not appear that the caption explaining the figure or how to extract information from it. Additionally, it is unclear how important it is for the reader to understand the estimated heuristic weights in tabular MDPs. Finally, the meaning of the $R^2$ values shown within the figure is not explained.\n\n- Figure 4 is not sufficiently descriptive. The arrows and annotations do not further clarify the context or content depicted, making it difficult to interpret the figure.\n\nQUESTIONS:\nPlease kindly visit the Strengths And Weaknesses. I do not have further questions.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces attention-aware inverse planning (AAIP), a framework for estimating people’s task-specific attentional bias from observed behavior by introducing an inverse problem of the value-guided construal model from previous work [1]. The authors (1) define the inverse problem of inferring bias parameters from state-action trajectories, (2) provide an MLE-based inference algorithm for this problem, (3) validate it in a tabular gridworld with accurate biases recovery performance, and show standard inverse reinforcement learning (IRL) fails to capture attention-limited policies, and (4) scale up to driving scenes simulated from the Waymo Open Dataset using a GPUDrive simulator and show accurate bias parameters recovery.\n\n[1] Ho et al., People construct simplified mental representations to plan, Nature (2022)\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. This paper clearly defines the attention-aware inverse planning problem and explains the motivation behind proposing this new problem (human behavior often has attention-related bias that cannot be captured by the standard IRL model). The problem proposed can be viewed as an inverse problem of bias parameters in the value-guided construal model from previous work [1].\n2. This paper conducted experiments in both tabular and scalable setups, showcasing the accurate biases recovering performance of the proposed algorithm. In the tabular case, it also shows that standard IRL fails to capture attention-limited policies, emphasizing the need of the proposed AAIP framework to accurately model agent behavior. \n\n**Weaknesses**\n1. The AAIP framework treats the reward as a fixed, known input rather than inferring it from data. In contrast, standard IRL recovers both reward and policy. Relying on a predefined reward limits AAIP's applicability in complex settings where the true reward structure is unknown or difficult to specify.\n2. The identifiability of bias in AAIP remains unclear. While the paper proposes a maximum‐likelihood estimator, it offers no analysis on when and to what degree the bias weights can be uniquely recovered from behavior. Empirically, both experiments explore only very simple bias spaces, so it is unknown how inference would perform in more complex settings. Even in the scalable Waymo experiment, the learnable bias parameter λ lies on a small discrete 1D space.\n3. There is no discussion of the algorithm’s time complexity, nor empirical runtime report. Such discussion would help clarify computing resource requirements and efficiency.\n4. No statistical significance info is provided for experiments. While authors argue that error bars and significance tests are unnecessary for this paper in the checklist, IRL literature typically reports results of multiple runs with different random seeds. Including such statistics would help readers understand the robustness of the reported bias‐recovery performance.\n5. Although the inference method is described in the text, there is no pseudocode or algorithm block to represent the overall inference procedure. A clear, step‐by‐step outline, which specifies inputs, outputs, and optimization steps, would make the procedure easier to understand. \n\nOverall, this paper introduces an interesting inverse problem of value-guided construal models that has potential in cognitive science research. However, I feel it is not yet ready for publication in its current form, given the limited scope of experiments and theoretical analysis.\n\nQUESTIONS:\n1. Given that AAIP currently assumes a known reward function, how might the framework be extended to infer both the reward and the attentional‐bias parameters λ when the reward is unknown or only partially specified?\n2. Could you provide the time complexity and empirical runtime of AAIP?\n3. Could you elaborate on why error bars and significance tests are unnecessary for experiments in this paper?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript introduces a framework called Attention-Aware Inverse Planning (AAIP) to interpret human cognitive limitations based on their behaviors. The core assumption is that people can’t plan with all the sensory inputs they receive from the external world due to perceptual costs, and thus they plan with a limited subset of objects in that state space. Unlike Inverse Reinforcement Learning (IRL), the framework does not assume a special form of reward function for these sub-state decision makers. The model aims to work backward from an observed behavior, such as a driving trajectory, to estimate the specific parameters of cognitive biases that have caused such limitations. The authors test this approach using simulated agents in a simple grid world and a more realistic driving simulator, showing that their method can successfully recover the built-in biases, unlike IRL.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1.  AAIP is  a new and important way to model behavior by inferring cognitive biases, moving beyond the limitations of traditional IRL, which might be crucial for understanding human behaviors and further design machines that interact with humans.\n2. The model is well-grounded in established cognitive theories like bounded rationality and value-guided construal, giving it strong theoretical validity and explanatory power.\n\n**Weaknesses**\n\n1. The paper's primary weakness is its exclusive reliance on synthetic data. Without real human data that includes humans' cognitive biases through self-report or eye movement, it’s hard to verify the explanatory power of the framework.\n2. The equation used to represent the value function assumes an addable state representation, which is not true in many real-world settings because the actual value in a task may rely on more than two-order level relationships of objects. This is also a core issue in modern DL interpretability studies where the inputs are high-dimensional, like images and videos. Therefore, I doubt if this framework can be applied to problems that are solved in an end-to-end fashion.\n3. The framework makes very simplified assumptions about decision makers’ attention, that they are simply selecting a fixed group of objects in the whole episode. This is a significant simplification that doesn't reflect the dynamic nature of attention in most real-world situations.\n\nQUESTIONS:\n1. In lines 214-215, the author used a “known” reward function for IRL, and with this reward design, the AAIP outperforms IRL in later experiments. However, I wonder if we can sample a reward function from more expressive reward function spaces; will things be different? Or, to put it more generally, I wonder if the deficit of IRL, which assumes full observation of states, can always be compensated by reward function design. Is there any bridge that can unify these two frameworks? Or is there an impassable divide between them? For example, can we borrow ideas from Partially Observable MDP to address the problem of vanilla IRL? These questions go far beyond the scope of the current paper, but since one of the core experiments in this study involves comparing two frameworks, I think a thorough discussion with some quantitative analysis would make the results far more convincing.\n\n2. Similar issues like Q1 regarding the specific form of cognitive costs and heuristic feature engineering exist in the manuscript. I expect more discussion on this in the final sections.\n\nLIMITATIONS:\n1. The study relies entirely on synthetic data and lacks validation with actual human behavior. The authors allude to this as a future step but don't frame it as a direct limitation of their current work.\n2. The framework cannot discover new cognitive biases; they must be defined by researchers beforehand. This makes it implausible in end-to-end cases where biases must be inferred automatically. Instead, the authors claim that more “complex heuristics” will be used in those cases, which I don’t see as a good choice for real-world problems.\n3. The model's focus is narrowly restricted to sub-optimality from inattention, ignoring other key factors in human decision-making, such as risk aversion/preference or memory constraints. The authors do not discuss this limitation.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper introduces attention-aware inverse planning, aiming to infer a person's attentional biases from their behavior. Unlike standard IRL, which assumes near-optimal decision-making, this work models how limited attention shapes choices using a parameterized bias function grounded in the value-guided construal framework. The authors also propose a scalable algorithm combined with deep RL and cognitive modeling to learn these biases via maximum likelihood. Experiments on show that this method outperforms IRL in capturing attention-limited behaviors in simulation experiments.

STRENGTHS AND WEAKNESSES:
I like the motivation of this work. Specifically, the authors address the gap between cognitive science findings and standard ML models. This is interesting to me especially about the realistic modeling framework grounded in human cognition. Besides, the proposed method provides a systematic, extensible approach for explaining, predicting, and influencing deviations from ideal rational behavior. Experiments also demonstrate the effectiveness of the proposed methods with outperforming standard IRL. 

However, the attentional heuristics used are relatively simple. The authors also mentioned this. Capturing more nuanced real-world biases may require richer heuristic models. Is there some better ways? Besides, although the method is claimed to be scalable and is demonstrated on the Waymo Open Dataset, the inference process—particularly the marginalization over all possible construals (as in Eq. 6)—can be computationally expensive in theory, raising concerns about scalability to more complex domains. It would be better for the authors to provide some latency analysis.

QUESTIONS:
What is the latency of the proposed method?

LIMITATIONS:
See weakness.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper highlights the impact of human cognitive biases and introduces attention-aware inverse planning, a method for inferring attentional biases from human behavior. It distinguishes this approach from standard inverse reinforcement learning and presents an algorithm that integrates deep reinforcement learning with cognitive modeling. Applied to driving data from the Waymo Open Dataset, the method uncovers drivers’ attentional strategies.\n\nThis paper makes a meaningful contribution to understanding and modeling human cognitive biases in decision-making. Despite some limitations in representation and modeling complexity, the framework is well-motivated, novel, and reasonably validated. I recommend acceptance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n+ Clarity and Motivation: The paper is well-written and clearly motivated, with a strong theoretical foundation and relevant citations.\n\n+ Novelty and Relevance: The focus on cognitive biases and limited attention in human decision-making is timely and important.\n\n+ Framework Design: The proposed framework is conceptually simple yet general, combining reinforcement learning components (reward, value, policy) with heuristics-based modeling of attentional biases.\n\n+ Scalability: Application to large-scale driving data showcases the method’s practical viability and relevance to real-world problems.\n\nWeaknesses:\n\n+ Simplified Construal Representation: The construal state is modeled as a set of objects, without accounting for inter-object relationships or spatial-temporal dynamics, which may limit expressiveness.\n\n+ Pipeline Complexity: The framework is not end-to-end; it relies on object detection as a preprocessing step to construct the construal space. This may introduce dependencies and limit generalizability.\n\n+ Heuristic Modeling: the heuristics used to uncover the cognitive biases are relatively simple and may not capture richer cognitive phenomena such as relational attention or dynamic prioritization.\n\nQUESTIONS:\n+ Could attentional biases be directly incorporated into the inverse reinforcement learning framework? For example, [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning, Zhang et al - CVPR 2020] model goal-directed attention using IRL.\n\n+ Is the construal mechanism conceptually related to personalized saliency or scanpath prediction? For instance, Xue et al. (Few-shot Personalized Scanpath Prediction - CVPR 2025) propose a method for few-shot personalized scanpath prediction, claiming the ability to learn individualized scanpaths with only a few episodes.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-06"}