Estimating cognitive biases with attention-aware inverse planning
Authors: Sounak Banerjee
Daphne Cornelisse
Deepak Edakkattil Gopinath
Emily Sumner
Jonathan DeCastro
Guy Rosman
Eugene Vinitsky
Mark K Ho
Venue: NeurIPS 2025 spotlight ()
Keywords: Inverse Planning
Bounded Rationality
Computational Cognitive Modeling
Inverse Reinforcement Learning
Human Interaction
Primary area: neuroscience_and_cognitive_science
TL;DR: We introduce the problem of attention-aware inverse planning, in which the goal is to estimate a resource-limited decision-maker’s attentional strategies from their behavior, and propose algorithms for solving this problem.

Abstract: People's goal-directed behaviors are influenced by their cognitive biases, and autonomous systems that interact with people should be aware of this. For example, people's attention to objects in their environment will be biased in a way that systematically affects how they perform everyday tasks such as driving to work. Here, building on recent work in computational cognitive science, we formally articulate the \textit{attention-aware inverse planning problem}, in which the goal is to estimate a person's attentional biases from their actions. We demonstrate how attention-aware inverse planning systematically differs from standard inverse reinforcement learning and how cognitive biases can be inferred from behavior. Finally, we present an approach to attention-aware inverse planning that combines deep reinforcement learning with computational cognitive modeling. We use this approach to infer the attentional strategies of RL agents in real-life driving scenarios selected from the Waymo Open Dataset, demonstrating the scalability of estimating cognitive biases with attention-aware inverse planning.

Estimating cognitive biases with attention-aware inverse planning
Sounak Banerjee l
Daphne Cornelisse!
Deepak Gopinath²
Emily Sumner?
Jonathan DeCastro" Guy Rosman Eugene Vinitsky
Mark K. Ho
New York University Toyota Research Institute
(sounak.
btounak.banerjee, banerjee, dc4971, vinitsky eugene, mark. Panyu. edu edu
{deepak booepakkgopinth, moily.sumnee, jonathan.decastz,, guy.rosman) (aammann@rriggoooo global
Abstract
People's goal-directed behaviors are influenced by their cognitive biases, and au- tonomous systems that interact with people should be aware of this. For example, people's s attention to objects in their environment will be biased in a way that sys- tematically affects how they perform everyday tasks such as driving to work. Here, building on recent work computational cognitive science, we formally articulate the attention-aven person's 's attentional biases from their actions. We demonstrate how attention-awan inverse planning systematically differs from standard inverse reinforcement learn- ing and how cognitive biases can be inferred from behavior. Finally, we present an approach to attention-aware inverse planning that combines deep reinforcement learning with computational cognitive modeling. We use this approach to infer the attentional strategies of RL agents in real-life driving scenarios selected from the Waymo Open Dataset, demonstrating the scalability of estimating cognitive biases with attention-aware inverse planning.
inverse planning problem, in which the goal is to estimate
Introduction
Conventional wisdom suggests that people not always rational. and this truism is now backed by over half a century of research in cognitive science [Simon, [953 Tversky and Kahneman 1974 Gigerenzer al. 2000]. It is remarkable, then, that standard approaches modeling goal- directed human behavior in machine learning and artificial intelligence (e.g., inverse reinforcement learning) assume that people act optimally- or least approximately optimally- with respect to their goals [Ng and Russell 2000 Arora and Doshi [202]] What explains this disconnect between psychological reality and basic modeling assumptions? How
bridge this gap?
One reason why machine learning models of human behavior assume japproximate-ootimaaltty is that although this assumption is false in fundamental ways, it underpins expected utility theory [yon] Neumann and Morgenstern 1947 McFadden] 1974). which itself provides the basis for developing scalable algorithms that can make generalizable predictions about human behavior (Ho and Griffiths 2022]. In contrast, general and scalable frameworks that account for why, when, and how people deviate from idealized rationality have not been previously available. This has changed in recent years: Building off of classic ideas in bounded rationality [Simon] 1990]. cognitive scientists have begun to go beyond just describing deviations from ideal rationality to systematically explaining, predicting, and influencing those deviations by developing general theories of resource- limited cognition [Lewis] al., 2014 Griffiths al., 2015 Gershman et al. 2013 Shenhav al. 2017 Bhui al.) [202]] The present work aims to apply these recent insights to the development of generalizable and scalable algorithms for modeling human decision-makinn -maiing.............
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Here, we focus specifically on how limited attention affects goal goal-directed behavior in complex tasks. As an example, consider the everyday activity of driving. When driving, people must systematically prioritize which objects in the environment to attend to (e.g., the car directly ahead, the child entering a blind spot) and which objects to ignore (e.g., the parked car on the opposite side of the street, the pile of leaves in driveway). Whether an object is prioritized for attention depends numerous factors, such as its immediate relevance for planning or a person's 's previous driving experience. Recent work by HH et al. [2022] has proposed the framework of value-guided construal, which formalizes the process of prioritizing attention for planning as a meta-cognitive decision. The value-guided construal framework can be viewed as a principled forward forward model of how unobservable cognitive processes (e.g., attention, planning, and biases) integrate to produce observable behavior.
In this paper, we extend the value- galue- construal framework to incorporate attentional biases and then invert it to specify the attention-aware inverse planning problem, in which the goal is to infer decision-maker from their behavior. We then study this approach
the biases of an attention-limite in tabular domains and compare standard modeling approaches that assume decision-maken have wnlimited attention (e.g., inverse reinforcement learning [Abbeel and Ng 2004]). Finally, we develop an approach for inferring biases from synthetically generated behavior in real-world driving
scenarios selected from the Waymo Open Motion Dataset [Ettinger et al. 2021], demonstrating the
feasibility of scaling up attention- aware inverse planning to complex, naturalistic domains.
2 Backkround
A number of algorithmic approaches have been developed for modeling human behavior. The simplest approaches are based on imitation learning [Bain and Sammut 1995). in which behavior is modeled by directly fitting a policy that predicts actions given observations. Imitation learning is effective when data is abundant and generalization is
and has been successfully applied to model human behavior in real-world tasks such as highway driving (Lefèvre et al. 2014 [Rudenko iet al. 2020]. However, when training data is limited or generalization is out-of-distribution it is necessary turn to methods that make stronger assumptions about policy structure. In particular, methods based on inverse reinforcement learning (IRL) assume that observed behavior is generated by a policy that optimal (plus decision- -noise) with respect to the true task dynamics and latent reward function, and attempts to infer the reward function Abbeel and N 2004 Ziebart et al.) 2008).
In human-robot interaction and AI safety, generalizable models of human cognition and action are key for safe and effective interaction [Collins et al.] [024). Although multi-agent interactions can be modeled without considering human biases via self-play or population-play algorithms (Silver eet al. 2017 Jaderberg et al., 2019), agents trained this way perform poorly with people, especially in cooperative tasks [Carroll et al. 2019]. Additionally, accurate representations of how people make decisions are needed for more sophisticated forms of interaction that involve reasoning about the physical capabilities and intentions of human users [Gopinath et al. [2011a]. modeling how humans reason about intentions (Dragan et al. [2013). or more complex forms of cooperation [Hadfield-M) Menelll al. 2016
et al. 20166. Separate from this work on behavior modeling, work on attention modeling in the machine learning community has mainly been pursued in the context of specific tasks, such as driving (Kotseruba and Tsotsos 2022, Gopinath et al.. 2021b Cao et al. 2022]. Although early work used cognitive models that relate attention to behavior [Salvucci] 2006]. more
recent work is based on data-driven approaches using gaze data and reaction times [Ahlström et al.) 2022
Sundareswara et al., 2013 Li et al. 2024 Yang et al., 2020 Xue et al., 2025)
IRL has also inspired computational approaches to studying human social cognition. For example, Baker et al. [2099] proposed that human theory of mind could be conceptualized as inverse planning,
Bayesian inference over a generative model of rational action. Additionally, as noted in the intro-
duction, a large literature documents human deviations from expected utility theory [Kahneman and
Tversky 1979]. More recent theoretical work (e.g., resource rational analysis [Grifliths et al., [013]) attempts to unify these disparate findings by explicitly modeling people's 's decision-making as a form of resource constrained optimization [Griffiths et al. 2015)] Gershman et al.) 2015). Along these lines, Ho et al. [022) proposed value-guide -uided construal, computational account of how people form simplified but useful world models when planning to make the best use of their limited attentional resources. Follow up work shows how learned biases (Ho et al. 2023) and properties of visuospatial
attention (da Silva Castanheira et al. 2025) interact with and bias value- guie-g construal. The present paper aims to build on these findings and apply them to behavior and attention modeling.
2

---

3 Attention-aware inverse planning
This section first reviews preliminaries for sequential decision-making and attention-limitee decision- making in the value-guided construal framework. We then describe an extension that incorporates attentional biases and invert this model to define the problem of atlention-aware inverse planning.
3.1 Preliminaries
We model sequential decision-makin
problems as Markov Decision Processes (MDPs), which consist
of tuple (S, A, R, 7), where S is a state space, A is an action space, T'; S A x S 00, 1 is transition function, R: : S x A + IR is a reward function, and E [0, 1) is a discount rate (Sutton and Barto 2018). Policies are mappings from states to distributions over actions, : S x A - [0, 1. The expected discounted future reward of following a policy π from a state s defines its value function:
RT) = E x -)
R(s,a) + IT(s' a.)) m)
a
An optimal policy is one that maximizes value at all states: x = arg max, Vss, rT, for all 8 E S.
Additionally, we assume object-oriented representation of states (Diuk et al. 2008). in which each state is a subset of objects O with features F that can take on values V that includes scalars and real vectors. That is, define the "sub-state space" corresponding to a subset of objects O S O the set of all maps from features of o E O to values, So = (O x F) + V. Then the full state space is their union: S Uogo So- For example, in driving setting, objects would correspond to vehicles (including the ego vehicle) or obstacles on the road. A state could then be represented as map, {(car_ 1_,, location) + (0.0, 1.0), (car_1,, s..eedd + 2.0, (car_2 location) H (5.5, 2.4).... We assume that if the transition function is defined for state with objects O then it is also defined for the abstracted state that only includes a subset of objects O' C O, their features, and their values.
3.2 Value-guided construals
The current work aims to characterize attention-aware inverse planning, and our first step towards this goal is to specify generative model of human planning in which attention plays a central role. To specify such a model, we build on existing work in computational cognitive science on resource- limited planning in humans, specifically the value suidedg construal approach, which provides a domain- -enerrll computational framework for how people form simplified mental representations when planning (Ho et al.) 2022 2023
In the value-guided construal approach, top-down attentional processes are modeled as selecting a subset of relevant details (c.g., objects and/or their affordances) that allow a person to plan an task. As simple but illustrative example, imagine trying to pass another car on the highway when there are two additional cars, one in the passing lane and one parked on the side of the road. Intuitively, the car being passed and the car the passing lane are relevant to planning, while the parked car is irrelevant. as a cost-benefit calculation: a construal of the situation that includes the cars on the road has a higher behavioral utility than one that only includes the car being passed because the latter leads to a plan that could lead to worse outcomes (e.g., potentially crashing into the car in the passing lane). Conversely, the construal with the car being passed and the car in the passing lane is also less representationally complex than one that includes all three cars, as it represents fewer entities, but it still has the same behavioral utility since the presence of the parked car does not affect optimal behavior. Thus, value-guided construal predicts that a rational, resource-limited trading off behavioral utility and representational complexity will form a construal that only includes the two cars on the road that is, they will only attend to those two cars.
effective solution
Value-guides construal formalizes this intuition
In the current work, we formalize the process of value-guided construal by controlling which objects are included in a state representation. That is, given a true state 8 composed of objects O, a task construal is subset of objects C' O. and the corresponding construed state is s(C] {o, f) + v E 8 | o E C), the subset of the object-feature to value pairs s that only includes the objects in C. Then, we define the construed policy the optimal policy where choices are made with respect
3

---

Ground Truth Task and Optimal Behavior
Construed Task and Behavior with Moderate Bias to Ignore Icc
Construed Task and Behavior with Strong Bias Ignore Ice
T.001
11000
0.08
RAZ
1.00
T.00
Figure 1: Attentional biases affect planned behaviors. (Left) A simple Driving World scenario in which an agent (blue car) receives + +10 reward upon reaching a goal location (blue square). Hitting a traffic cone results in -10 reward, hitting parked car results in -100 reward and termination, and moving on ice leads to slipping the left right with probability 0.4 (see main text for full domain specification). Samples from the optimal policy (n = 100, blue lines) avoid both ice patches. (Center) Attention-an aware inverse planning assumes that Decision-maker's behavior reflects the formation of simplified task construals that only include telk-relevant details [Ho et al. 2022]. In the current scenario, decision-maker who also has moderate bias to ignore ice (Aice the first ice patch but not the second. Shown is the average construal and trajectories that the agent expects to occur (that is, does not consider potentially slipping on the first ice patch). Numbers indicate the marginal probability of attending to an object. (Right) A decision-maker with strong bias to ignore ice (Aic = - 100) will drive over both ice patches and risk hitting the parked cars. For both, ACone 10 and Apared 0.
10) will drive over
to the construed state: Te(a s) = (a s(C] The value of representation associated with construal is then a sum of two terms:
VOR(s, C) V(s, TC) Cost(C)
The first term the behavioral utility of a construal, the actual value of acting optimally from state as if the construed state were the case, The second term is the cognitive cost of the construal, which we assume simply the negative of the cardinality of the construal (i.e., the number of objects represented) as in previous work Ho et al. 2022].
Note that while construals are parameterized sets of objects, the resulting construed policy Te incorporates all interactions between construed objects. This is because the policy is associated with value function that is derived from construed reward and transition functions for the task:
Qo(s,a) Ro(s, a) + ETols (o)(s)
(3)
The reward function Rc) R(s(C],a) and transition function To(s' s, = T(s'(C)
*(C), incorporate the interactions between objects in C' (and nothing else). This means the value functions and policy (i.e., Vc. Qc and mc) also incorporate information about interactions.
Previous work on value-guided construal focused on simple discrete domains to ensure tight experi- mental control for human studies. Here, we extend this work in three distinct ways: (1) we incorporate heuristic biases in the construal selection policy. in Section 3.3 (2) we articulate the iitention-awary inverse planning problem, in which the goal is infer attentional biases from behavior, in Section 3.4 and (3) we validate the approach in a tabular domain as well as in driving simulator with continuous states, Section
TTis formulation of construed policy terms of masking states passed to a general optimal policy is roughly equivalent previous parameterizations (e.g., [H et al. )2022)), but allows us to use single pre-traine trainra policy network instead of explicitly computing separate construed policies, described in Section 4.3

---

3.3 Modeling biased construal formation
The behavioral utility a construal formalizes top-down, goal-directed influences on attention, but other factors can influence what construals person will form. For example, some objects are highly salient due to low-level perceptual features, such as an orange traffic cone or red sports car. Other objects are more or less salient because of heuristics that have been learned from previous experiences in similar tasks. For instance, someone who has extensive experience driving in North Dakota might learn that, by default, with important consequences for driving. In contrast, someone who primarily drives in southern California may have weaker bias for attending to ice even bias for ignoring ice.
should attend ice the road since that has been a frequent type of entity
We can extend the original value-guided construal framework to incorporate heuristic strategies by adding parumeterized bias function, Hx(s, C¹) that weights a construal C' in state and is parameterized by X. A person's construal selection policy at a state s is then a softmax distribution over the value of representation and the bias [Ho et al. 2023
x xC 8) x VOR(ACC)+H
At the beginning of an episode, TA 8) and then executes its corresponding policy, TC, in the environment. Note that here, we do not model dynamic construals or switching construals within an episode, but we return to these factors as important directions for future work in the discussion.
decision-maker samples construal according to
3.4 Inferring heuristic biases from behavior
decision making as a form of biased value-guided construal, we turn to main focus: Can we estimate attentional biases from behavior? Of course, in principle, given enough data and an accurate model of a data- generatinn process, one can estimate the parameters of any stochastic process up noise factor. However, it is unclear what this looks like in practice in the context of attention-awars inverse planning. Under what conditions are heuristics going to be identifiable? How much behavioral data is sufficient to obtain accurate estimates? Our goal is to provide preliminary answers to these questions.
Having specified attention-limin
Formally, estimating heuristics from behavior can be framed as probabilistic inference problem. Given a state-action trajectory Ç = ((#1, 01): (1;;... (ST, ar)), maximum-likellhee attention-waree inverse planning involves finding the set of heuristic bias parameters X that maximize the probability of the observed behavior:
X = arg max P(C AA)
(5)
arg max
Σ P(C CC)
(6)
Σ
II ma st) Tx (1)
= arg max
The maximand in Equation 5 can be computed if the space of construals is enumerable and we can compute Tc- For our experiments in a tabular domain, we calculate these quantities exactly using dynamic programming in JAX [Bradbury et al., 2018) and then solve for X using off-the- shelf optimizers [Virtanen et al.) 2020 For experiments involving real-world driving scenarios, we approximate them by using pre-trained policy network to amortize the computational overhead of planning and masking states to only include subsets of objects.
4 Experiments and Results
We conduct three experiments to validate the attention- inverse planning approach to behavior modeling. In Section 4.1 we show how it can be used to infer heuristics and attentional biases
from behavior in tabular setting. In Section 4.2 we provide evidence that, unlike our proposed
approach, standard approaches like inverse reinforcement learning [Abbeel and Ng. 2004) are
not guaranteed to recover citectioonlimitee decision-making Finally, in Section 4.3 describe
5

---

Ice Heuristic Weight
Cone Heuristic Weight
Parked Car Heuristic Weight
25
25
25
Estimated
Iestimated
Eistimatedd
x XX
-25
-25
-25
R² = 0.77
=
0.76
R² = 0.37
50
P 1.0E
04
-50
P 1.0E
04
-50
P 1.0E 04
50
-50
-25
50
-50
25
50
True
True
True
Figure 2: Results of joint maximum-likclihonn attention inverse planning: Actual weights of heuristic biases plotted against model estimates, in Driving World (tabular MDP). We simulated the behavior of 1000 agents (with different combinations of weights for three heuristics) by sampling 125 trajectories from 25 different WrivingWorld scenarios (5 from each scenario), for each agent. Then, attempted to learn the weights jointly from behavior via maximum miikiimu--ikellhoodd likelihood estimation. Each sample represents the true weight (x-axis) of a heuristic for an agent and its estimated value (y-axis) derived from the agent's behavior. Correspondence between true and estimated weights demonstrates that heuristics that are less consequential (e.g., a bias to ignore cones) are more readily identifiable from behavior (indicated by higher R² values). In contrast, biases that could be consequential (e.g., a bias to ignore parked cars) are less identifiable because they are outweighed by the effect of value-guided construal. These results illustrate the viability and challenges of attention-aware inverse planning in tabular setting.
an approach to attention-aware inverse planning in a high-fidelity driving simulator using real- world scenarios. Code for replicating the experiments discussed in this section is available at: CoDerivvee--oee//eeeeeeee New.eeeeeeurSSS22222
4.1 DrivingWorld
To study how attentional biases affect planned behaviors in a simplified setting, we implemented the tabular Driving World World domain shown in Figure [ The agent controls the blue car and can take the following actions on each timestep: move up cell (-1 reward), move up cells (-1 reward), move diagonally left (-2 reward), and move diagonally right (-2 reward). A scenario consists of parked cars, ice patches, traffic cones, and a goal state. The episode terminates if the agent reaches the goal (-100 reward), hits parked car (-100 reward), hits wall (-100 reward) or leaves the grid (0 reward). Driving into a traffic cone is penalized (-10 reward). Finally, there is no additional reward for driving on ice, but with probability 0.4 the slips either to the left or right and may inadvertently hit another object.
To apply our model of attention-limited bias function HA using a fixed set of construal feature functions, {ői). that map construals and states to real values and parameterize it with linear weights {A,]. such that Hs(s, C*) E, A, (s, C). For example, a bias to attend to or ignore ice patches captured with a weight Atce R and feature function (s, C) that returns the number of ice patches in C' that are visible in state We assume corresponding weights and feature functions for traffic cones (Acomer &come) and parked cars (Aparedd Orarked). To illustrate the effect of different biases, Figure I compares the optimal, iitention-urlimiin behavior in the task to the behavior of attention-liminn moderate bias to ignore ice (Aiee -00 and a strong bias to to ignore ice (Aice 100)..
maciiion-making to Driving World scenario, we define the
decision-makers with a
Inferring heuristic biases. To validate our approach for attention- inverse planning in WoivingWorld, we apply the procedure synthetic dataset generated by first sampling 1000 values of Åice- ACone- and Apased uniformly sampled between -50 and 50, and then sampling 125 construals and trajectories across the 25 scenarios (see Appendix (A). We then computed the joint maximum likelihood heuristic weights, (Xice >ooney Aparked) ). A comparison of the true and estimated values (Figure 2 shows that some parameters can be robustly recovered across range of values (e.g., Atee and ACone) whereas others more difficult to recover accurately (c.g., Aparked)

---

Attentionllmitod docisiou-maker policy
Inferred IRL policy
fise
Moc+Coone
?
NLL
Noise
X
555
B
IRL
X
X
X
537
X
X
X
537
X
X
X
X
X
513
AAIP
X
X
NA
NA
NA
265
(A)
(B)
Figure Inverse reinforcement learning (IRL) is not guaranteed recover attention ditienttion-immteed decision- making. (A) We simulated ground-truth iveetion-limitee decision-make with bias ignore ice (Atee = 10) and generated 100 trajectories the scenario above. This data was passed to an IRL procedure in which we fit the action inverse temperature (3), random action selection (s), auxiliary reward for driving on ice (fice), auxiliary reward for driving on ice between cones (fice+Cone), ) and discount rate (y). We simulated 100 new trajectories in the same scenario with a policy using the fit parameters. Note that compared to the original policy, the resulting IRL policy is noisier and visits different ice patches. (B) As quantitative evaluation, compared fitted negative log- ligeliicls (NLL) for policy given optimal action values with fitted decision-noiss parameters /3 and e (Noise), three increasingly expressive versions of our IRL procedure, and attention-aware inverse planning (AAIP). Noise and AAIP are intended to serve as a ceiling and floor on negative log-likelihoods, respectively. Even when 8, E, TTee ConesCone and 7 are fit, IRL has limited ability capture the data. In the table, x indicates that parameter was fit, otherwise it was set to its true value.
4.2 Comparison with inverse reinforcement learning
Attention-awerr inverse planning generalizes models of inverse decision-making, but natural question to ask whether existing approaches can capture regularities in behavioral data even they mis-spy the data-generating process (c.g., assume people not diientionllmmiees when in fact, they are). Inverse reinforcement learning (IRL) typically assumes that an expert acts based accurate and complete representation of the task dynamics but optimizes unknown reward function (Abbeel and Ng. 2004). IRL aims to infer this unknown reward function. Crucially, even though IRL applied to human-like behavior (incorrectly) assumes a pocisiionmakee has unlimited attention, a sensible reward function may always exist that sufficiently captures human behavior.
To demonstrate that this is not always the case, we identified a simple DrivingWorld scenario (Figure A which inferring rewards not sufficient to capture the behavior of an attention- limited policy. We first generated 100 trajectories from an attention-limite policy with Åtce 100, ACone = 10, and Apaked = To simplify the IRL problem, we assumed that the true reward function known (hitting cones and parked cars is penalized, reaching the goal is incentivized) but that there is an unknown auxiliary reward added to the true reward. This auxiliary reward added a bonus Pice IR for visiting any ice patch and flse+Come E R for visiting ice patch in between cones. We then calculated a combination of the auxiliary rewards parameters, softmax inverse temperature, random choice, and discount rate that maximizes the liglliieldood of the generated data.
As shown in Figure the IRL policy that best captures data misses out on several key properties of the original policy. First, it is much noisier and sometimes even fails to reach the goal altogether. Second, whereas the original policy visits the two patches between the cones in roughly equal proportion, the IRL policy only visits the first patch and largely avoids the second. We further confirmed that the IRL policy substantially deviates from the original policy with a quantitative comparison of the negative log- ligelikeisoods (Figure 3B).
4.3 GPUDrive: A high fidelity driving simulator with real-world data
Next, we turn to methods for scaling up to more complex, continuous domains. Specifically, we test an approach to computing the quantities required for attention aware inverse planning that involves

---

Full state space:
Construed state sattesspcc: near-optimal behavior
Construed state-space: sub-optimal behavior
optimal behavior
Car
Car behind
Start Positin (Ego)
Trajectory (Ego)
Goal Position (Ego)
Trajectory (Construa) vehicle)
Figure 4: This demonstrates the driving behavior of PPO agent controlling the ego vehicle in Waymo highway scenario under three conditions: (i) when it can see every vehicle in the scene (left; optimal); (ii) when it can see single vehicle ahead, that it needs to pass before reaches the goal (middle; near-optimal); and (iii) when it can see single vehicle at its rear that has impact on the agent's plan (right; sub-optimal). The trajectories, in translucent red, represent the hehavior of the ego vehicle controlled by the PPO agent across 40 trials in each condition. Solid lines of all other colors represent the trajectories of other vehicles the scene. The purple solid lines the middle and right plots show the trajectories of the single vehicles being observed in each of the two conditions. The behavior of our PPO agent (distribution of ego trajectories) the optimal (left) and near-optimal (middle) conditions similar, where the agent goes around the vehicle ahead. In the sub-optimal condition (right), the agent attempts to drive straight towards the goal position, resulting in a crash.
passing masked states, s(C],, to pre-trained generalist policy representing x in order to obtain and evaluate construed policies TO. This approach effectively amortizes the cost of computing construed policies, enabling us to scale up more complex and naturalistic tasks, such as driving.
For our experiments, we use the GPUDrive driving simulator (Kazemkhani et al. 2025] to generate synthetic trajectory data in scenes obtained from the Waymo Open Motion dataset [Ettinger et al.) 2021]. Vehicle trajectories were first generated using a biased generative model parameterized by heuristic bias weights (Ar). We then showed that it is possible to retrieve the true Arr, by performing inference over the synthetic trajectory data, as described equation [ We describe the synthetic
data generation and inference processes greater detail in Scctions 4.3.1 and 4.3.2 respectively.
4.3.1 Synthetic data generation
Ten scenarios were manually selected (illustrations in Appendix B from the Waymo Open Motion dataset, where the movement trajectory of the agent-controlled (ego) vehicle presented high degrees of interaction with other vehicles in the scene. As a result of our selection process, a large portion of our simulation dataset contains scenarios where the ego vehicle yields to other vehicles, merges into highways, or changes lanes on multi-lane roads. These are all cases where failing to attend to the relevant objects can lead to catastrophic outcomes (crashes).
Behavioral Utility. For each scenario, we computed the behavioral utility, Vs, TT), of all possible single-vehicke construals (any one vehicle is observed in each construal) using Monte- Carlo estimation, The behavioral utility construal was computed by taking the average reward earned by a construed policy- the generalist policy that received masked (i.e., construed) state that only included a subset of the vehicles from the true state-across across N rollouts in the true state (algorithm pseudocode available in Appendix D). For example, if there were 15 vehicles in scene, then 15 different construed models

---

Deviation from Ego Collision
Deviation from Ego Heading
Re Heading
20
15
15
Predicted Weight
10
Predicted Weight
Predicted Weight
-10
R2 = 0.77 Pic. 1.0E 1000---0 04
=
0.76
A² = 0.83 P 1.0E 04
Pix 1.0E - 04
22
155
-20
-10
10
20
-15
15
-15
-5 Artual Weight
15
Weight
Artual Weight
Figure 5: Attention-aware inverse planning results with Waymo Open dataset scenarios in GPU- Drive [Eutinger et al., 2021 Kazemkhani et al., 2025]. We simulated 215 agents with different heuristic weights by sampling 80 trajectories from 10 scenarios per agent. Maximun-likelihood weights were calculated using Bayesian Optimization [Nogueira 2014). Each points represents the true weight of heuristic for an agent and its estimated value derived from the agent's behavior. Cor- respondence between true/estimated weights demonstrates that this approach can recover underlying agent biases in complex domains such as real-world driving scenarios.
were each rolled out N times, and the behavioral utility of each construal was computed as the average reward of the construed model across these rollouts:
N T())
1 N
((+ x)=
Σ R(s
a
(8)
The value of representation was then obtained for each construal based on Equation
Heuristic Biases We computed a construal selection policy, (C s). that incorporated three heuristics encoding information about the relative motion of objects, which are features that have been shown to be associated with hazard perception in driving [Lacherez et al.) 2014]. The biases were: (i) Deviation from Ego Heading (DfEH), (ii) Relative Heading (RH), and (ii) Deviation from Ego Collision (DfEC) (also see Appendix C The heuristics were computed at the beginning of each episode. similar to the procedure described in Section B.3 To perform inference over the heuristic weights of agents for the 10 scenes, we needed ground-truth data containing state information and corresponding agent actions across entire trajectories. However, the Waymo dataset only provides state representations. So, we used a deep-RI. PPO model [Cornelisse et al. 2025) pre-trained on GPUDrive as a generalist model generate sample state-action trajectories {{} in the simulator. All other vehicles in the scene followed the logged (human) trajectories from the dataset.
Generative Model To generate trajectory dataset that represents the behavior of agent with specific set of parameter values for the three biases (ADEH ARH. Apee). a biased construal selection policy for the specific set of X-values was first used to sample eight construals (with replacement) from each of the 10 scenes. Finally, state-action trajectories were generated by rolling out our generalist policy on construed (masked) state-spaces, based the sampled construals for each of the 10 scenes. This 80-trajectory dataset then served as the synthetic 'ground-truth" behavioral data. 80 trajectories equate to 12 minutes of driving data, which was found be the minimum amount of data necessary for our algorithm to converge (more details in Appendix G). All simulations were run on an Ubuntu High Performance Computing cluster. single RTX8000 GPU and 40GB of memory were allocated for the process.
4.3.2 Results: inferring heuristic parameters
To validate our approach, we performed a procedure similar to that described in Section 4.1 We uniformly sampled 215 sets of X-values for the three heuristics, within reasonable range of values for each heuristic. For each set, we then sampled trajectories using the generative model described above. Finally, performed maximum likelihood estimation using Bayesian optimization [Nogueira

---

The results of our inference process are presented in Figure which demonstrate that our algorithm for attention-aware inverse planning can reliably recover underlying heuristic biases of simulated agents in real-world scenarios. Details about program execution latency for the current experiment can be found Appendix F
2014 to recover the true As (algorithm available in Appendix E
Limitations and future work
We study attention-aware inverse planning and show how it can scale to continuous, naturalistic domains such as driving, but there are limitations to the current work. First, we constrain the decision- maker to a single fixed construal, chosen at the beginning of an episode, rather than modeling dynamic changes in construal. This can be addressed in future studies by incorporating dynamic updating of task construals in response to task-related events, such as the sudden appearance of new objects. Second, we focus exclusively on how construal leads to systematic deviations from idealized rationality, but people's decisions could be shaped by variety of other cognitive biases [Liederr and Griffiths 2020 Schulze et al. 2025 or perceptual mechanisms (Yang et al. 2020]. Future work will need to incorporate these factors. Third, the current work assumes a preciiee set of possible attentional heuristics, which can be useful for interpretability and data efficiency. However, in situations where interpretability is not priority or domain knowledge is limited, but behavioral data is abundant, it will be important to extend our approach to learning parameterized heuristics.
Additionally, our work could be extended to incorporate joint inference about rewards and attentional biases since here we assume known reward model. Previous work in cognitive science has explored how people make these joint inferences [Rane et al. 2023], and theoretical work on "robust IRL" has shown that mis-specification of an expert's dynamics model can lead to systematic error between the value between policies estimated by IRL and the expert policy (Xu and Liu 2024]. Indeed. the results of section 4.2 suggest that it not difficult find scenarios where this error meaningful and that the recovery gap may be significant in practice. Finally, we apply attention-awar inverse planning in real-world scenarios taken from the Waymo Open Dataset but estimate heuristies from synthetic data. While this approach provides insight into the feasibility of estimating heuristics in complex tasks, validation of our algorithm on human behavior in more tasks an important next step.
Conclusion
This paper aims to bridge recent insights on resource-limited decision-makine from cognitive sci-
ence |Griffiths et al. 2015 Gershman al.) 2015 Lieder and Griffiths 2020 with scalable human
behavior modeling Lefévre et al. 2014] Rudenko et al. 2020 Abbeel and Ng 2004 Ziebart et al.
2008). Specifically, we extend the framework of value- value-guides guided construal [Ho et al. 2002|[023]
formal account of how people allocate limited attention during planning, and articulate the attention- aware inverse planning problem, in which the goal is to estimate, from a person's 's actions, what attentional biases influence their behavior. We conducted two sets of experiments to validate this approach to behavior modeling. Our first set of experiments demonstrated the fcasibility of estimating cognitive biases from data in a tabular setting and show that standard inverse reinforcement learning can sometimes fail to capture the behavior of -imined decision-makers. Our second set of experiments further validate this approach in a high-fidelity driving simulator [Kazemkhani et al. 2025) using real-world scenarios taken from the Waymo Open Dataset [Ettinger al.] 2001].
Aside from demonstrating the feasibility of scaling up attention awaar inverse planning to complex domains such as driving, the current work shows how principled modeling approaches from computa- tional cognitive science [Griffiths et al. 2024) can integrate effectively with modern machine learning methods (e.g., deep RL.. In doing so, contributes both to the development of safe, interpretable, and trustworthy systems that interact with people [Ho and Griffiths [022] as well as to the creation of generalizable cognitive models and theories that can capture the full range of natural human behavior [iise et al. 2024 Carvalho and Lampinen] 2025).
Acknowledgements
This work was supported by NSF Award #2348442 and the TRI University Research Program.
10

---

References
Pieter Abbeel and Andrew Y. Ng. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the s1st International Conference on Machine Learning, ICML '04, page New York, NY. USA, 2004. Association for Computing Machinery. ISBN 158 15811 138385. doi: 10.1 14/1101153300101440000
Christer Ahlström, George Georgoulas, and Katja Kircher. Towards context-dependent multi-buffer driver distraction detection algorithm. IEEE Transactions on Intelligent Transportation Systems, (3(5):4778-47999 2022. doi: 10. 10210997...2222133000
Saurabh Arora and Prashant Doshi. A survey of inverse reinforcement learning: Challenges, methods and progress. Artificial Intelligence, 297:103500, August 2021. ISSN 0004-3702. doi: 10.1016/j.
Michael Bain and Claude Sammut. A framework for behavioural cloning. In Machine intelligence 15, pages 103-129, 1995.
Chris L. Baker, Rebecca Saxe, and Joshua B. Tenenbaum. Action understanding as inverse planning. Cognition, 113(3):22--49, December 2009, ISSN 0010-0277. doi: 10.1016f..oognition200999 005.
Rahul Bhui, Lucy Lai, and Samuel Gershman. Resource-rational decision making. Current Opinion in Behavioral Sciences, 41:15-21 21, 2021. ISSN 2352-1546. doi: 10.1016/f.co cobeha.2021..02.000
James Bradbury. Roy Frostig. Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL
http://githut com/ jax ml/jax
Zhangjic Cao, Erdem Biyik, Guy Rosman, and Dorsa Sadigh. Leveraging smooth attention prior for multi-agent trajectory prediction. In International Conference on Robotics and Automation, pages 10723-10730. IEEE, 2022.
Micah Carroll, Rohin Shah, Mark K. Ho, Thomas L. Griffiths, Sanjit A. Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-Al coordination. Curran Associates Inc., Red Hook, NY, USA, 2019.
Wilka Carvalho and Andrew Lampinen. Naturalistic computational cognitive science: Towards generalizable models and theories that capture the full range of natural behavior, 2025. URL 20349
htttps://arxiv.org
Katherine M. Collins, Ilia Sucholutsky. Umang Bhatt, Kartik Chandra, Lionel Wong, Mina Lee, Cede- gao E. Zhang. Tan Zhi-Xuan, Mark Ho, Vikash Mansinghka, Adrian Weller, Joshua B. Tenenbaum, and Thomas L. Griffiths. Building machines that learn and think with people. Nature Human Behaviour, 8:1851-1863, October 2024. ISSN 2397-3374. doi: 10.1138/441552-044--1999.9
Daphne Cornelisse, Aarav Pandya, Kevin Joseph, Joseph Suárez, and Eugene Vinitsky. Building reliable sim driving agents by scaling self-play, 2025. URL https: /ttp:://axxviiiiie wrg..bs///bb//2/a/ 14706
Jason da Silva Castanheira, Nicholas Shea, and Stephen M. Fleming. How attention simplifies mental representations for planning, 2025. URL
http///awww..rr/////////../../// 09520
Carlos Diuk, Andre Cohen, and Michael L. Littman. An object-oriented representation for efficient reinforcement learning. In Proceedings of the 25th International Conference on Machine Learning, ICML "08, page 240-247, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781605382054,
10.1145/1390156. 1390187.1
Anca Dragan, Kenton CT Lee, and Siddhartha Srinivasa. Legibility and predictability robot motion. In International Conference Human-Roun Robot Interaction, pages 301-308. IEEE, 2013.

---

Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp, Charles R. Qi, Yin Zhou, Zoey Yang, Aur'elien Chouard. Pei Sun, Jiquan Ngiam, Vijay Vasudevan, Alexander McCauley, Jonathon Shlens, and Dragomir Anguelov. Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. In IEEE/CVF International Conference on Computer Vision, pages 9710-9719, October 2021.
Samuel J. Gershman, Eric J. Horvitz, and Joshua B. Tenenbaum. Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science, 349(625), 2015. doi: 10.1 wtt////iennc..ac//////
Gerd Gigerenzer, Peter Todd, the ABC Research Group, et al. Simple heuristics that make us smart. Oxford University Press, 2000.
Deepak Gopinath, Mahdieh Nejati Javaremi, and Brenna Argall. Customized handling of unin- tended interface operation in assistive robots. In IEEE International Conference on Robotics and Automation, pages 10406-10412. IEEE, 2021a.
Deepak Gopinath, Guy Rosman, Simon Stent, Katsuya Terahata, Luke Fletcher, Brenna Argall, and John Leonard. MAAD: A model and dataset for" attended awareness" in driving. In IEEE/CVF International Conference on Computer Vision, pages 1426-3436, 2021b.
Thomas L. Griffiths, Falk Lieder, and Noah D. Goodman. Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic. Topics in Cognitive Science, 7(2): 217-229, April 2015. ISSN 7756-8757. doi: 10.11 10.111
Thomas L Griffiths, Nick Chater, and Joshua B Tenenbaum. Bayesian models of cognition: Reverse engineering the mind. MIT Press, 2024.
Dylan Hadfield-Menell, Stuart Russell, Pieter Abbeel, and Anca Dragan. Cooperative inverse reinforcement learning. Advances in Neural Information Processing Systems, 29, 2016.
Mark K. Ho and Thomas L. Griffiths. Cognitive science as a source of forward and inverse models of human decisions for robotics and control. Annual Review of Control, Robotics, and Autonomous Systems, 5(1):33-53, May 2022. ISSN 2573-5144, doi: 10.1
Mark K. Ho. David Abel, Carlos G. Correa, Michael L. Littman, Jonathan D. Cohen, and Thomas L. Griffiths. People construct simplified mental representations to plan. Nature,
May 2022. ISSN 1476-4687. doi:
Mark K. Ho, Jonathan D. Cohen, and Thomas L. Griffiths. Rational simplification and rigidity in human planning. Psychological Science, 34(11): 12221 October 2023. ISSN 1467-9280. doi: 10.1 00.117770956777621225555
Max Jaderberg, Wojeiech M. Czarnecki, lain Dunning, Luke Marris, Guy Lever, Antonio Garcia Cas- tañeda, Charles Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Leibo, David Silver, Demis Hassabis, Koray Kavukcuoglu, and Thore Graepel. Human-level performance in 3d multiplayer games with population- based reinforcement learning. Science, 364(6443):859 865, 2019. doi: 10. 1226/science.aau666
Daniel Kahneman and Amos Tversky. Prospect theory: An analysis of decision under risk. Æcono- metrica, 47(2):263, March 1979. ISSN 0012-9682, doi: 10.2307/1914185.
Saman Kazemkhani, Aarav Pandya, Daphne Cornelisse, Brennan Shacklett, and Fugene Vinitsky. Gpudrive: Data-driven, multi-agent driving simulation at million fps. In International Conference on Learning Representations (ICIR), 2025. URL https: //ar:iv......../a/aaa http//axxvvvrgg./e/// arg/abs//n /408. 2408.00184
Max Kleiman-Weiner Mark K Ho, Joseph I. Austerweil, Michael L. Littman, and Joshua B Tenen- baum. Coordinate cooperate compete: abstract goals and joint intentions in social interaction. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 38, 2016.
Tuliia Kotseruba and John K Tsotsos. Attention for vision-based assistive and automated driving: A review of algorithms and datasets. IEFE Transactions on Intelligent Transportation Systems, 23
(119:19907-19928. 2022.
12

---

Philippe Lacherez, Sandra Au, and Joanne M. Wood. Visual motion perception predicts driving hazard perception ability. Acta Ophrhalmologica, 92(1):88 93, 2014. doi: https://doi.org/I1 111755-5768.2212..257777
Stéphanie Lefévre, Dizan Vasquez, and Christian Laugier. A survey motion prediction and risk assessment for intelligent vehicles. ROBOMECH Journal, 1(1), July 2014. ISSN 2197-4225. doi: 10.1 10.1186//40688.. 000-4006660411000-----10 014-0001-7.
Richard Lewis, Andrew Howes, and Satinder Singh. Computational rationality: linking mechanism and behavior through bounded utility maximization. Topics in Cognilive Science, 6(2):279-311, April 2014. ISSN 1756-8765. doi: 10. 1/tops 12086
Chenran Li, Aolin Xu, Enna Sachdeva, Teruhisa Misu, and Behzad Dariush. Optimal driver warning generation in dynamic driving environment. In IEEE International Conference on Robotics and Automation, pages 14184-14190. IEEE, 2024.
Falk Lieder and Thomas L. Griffiths. Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources. Behaviorul and Brain Sciences, 43:e1., February 2020. ISSN 1469-1825. doi: 10.1017/S0140525XX 190000
Daniel McFadden. Conditional logit analysis of qualitative choice behavior. In Paul Zarembka, editor, Fontiers in Econometrics, pages 105-142. Academic press, New York, 1974.
Andrew Y. Ng and Stuart J. Russell. Algorithms for inverse reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, ICML '00, page 663-670, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc. ISBN 1558607072.
Fernando Nogueira. Bayesian Optimization: Open source constrained global optimiza-
tion tool for Python, 2014. URL bstps://githut http:///ttuuu..coooaayeeesaaaaae comm/aayyeiiannnopiiiizaaiio
anQptimizatioon
Sunayana Rane, Mark Ho, Ilia Sucholutsky, and Thomas L. Griffiths. Concept alignment as a prerequisite for value alignment, 2023. URL battps:/aatt awxvv orw/ooo/////////////oo/eee abs/aaaaaa00 20059
Andrey Rudenko, Luigi Palmieri, Michael Herman, Kris M Kitani, Dariu Gavrila, and Kai O Arras. Human motion trajectory prediction: A survey. The International Journal of Robotics Research, 39(8):85-9935, 2020.
doi: 10. 10.11770077336992011444
Dario D. Salvucci. Modeling driver behavior in a cognitive architecture. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48(2):362-380, June 2006. ISSN 1547-8181. doi: 10.15 (0001111000077277777441111
Christin Schulze, Ada Aka, Daniel M. Bartels, Stefan F. Bucher, Jake R. Embrey, Todd M. Gureckis, Gerald Häubl, Mark K. Ho. Ian Krajbich, Alexander K. Moore, Gabriele Oettingen, Joan D.K. Ongchoco, Ryan Oprea, Nicholas Reinholtz, and Ben R. Newell. A timeline of cognitive costs in decision-mann making. Trends Cognitive Sciences, 29(9):227-88 839, 2025. ISSN 1364-6613. doi: https://doi.org/10 00. t/..tt.iics.........o....oo.l w0t....022.0000000
Amitai Shenhav, Sebastian Musslick, Falk Lieder, Wouter Kool, Thomas L.. Griffiths, Jonathan D. Cohen, and Matthew M. Botvinick. Toward rational and mechanistic account of mental effort. Annual Review of Neuroscience, 40(1):99-124 July 2017. ISSN 1545-4126. doi: 10.1 146/ manurev-neuro-0222 17-116-315..
David Silver, Thomas Hubert, Julian Schrittieserr, loannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Si- monyan, and Demis Hassabis. Mastering chess and shogi by self-play with general reinforcement learning algorithm, 2017. URL http://///axvvv.r////// /ttp////1111112111111
Herbert A. Simon. behavioral model of rational choice. The Quarterly Journal of Economics, 69 (1):99, February 1955. ISSN 0033-5533. doi: 10.2307/1884852.
Herbert A. Simon. Bounded Rationality, page 15-18. Palgrave Macmillan UK, 1990. ISBN 9781349205684, doi: 1-111997881149-22568--444444
13

---

Rashmi Sundareswara, Mike Daily, Mike Howard, Howard Neely, Rajan Bhattacharyya, and Craig Lee. Using distracted driver's behavior inform the timing of alerts in comi-autonomous car. In 2013 IEEE International Milti-Disciplinary Conference Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pages 199-202. IEEE, 2013.
Richard Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT Press, 2018.
Amos Tversky and Daniel Kahneman. Judgment under uncertainty: Heuristics and biases. Science, (55(4157)1124-111 1131,, 1974. doi: 10.1 10.1126/science.18555. 185.4157.1....
Pauli Virtanen, Ralf Gommers, Travis Oliphant, Matt Haberland. Tyler Reddy, David Cournapeau. Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. Nelson, Eric Jones, Robert Kern, Eric Larson, C Carey, ilhan Polat, Yu Feng. Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SeiPy 1.0: Fundamental Algorithms for Scientific Computing Python. Nature Methods, 17:261-272. 2020. doi: 10.
John von Neumann and Oskar Morgenstern. Theory of Games and Economic Behavior, 2nd rev. Princeton University Press, Princeton, NJ, USA, 1947. ISBN 978-1-40082946-0.
Toby Wise, Kara Emery, and Angela Radulescu. Naturalistic reinforcement learning. Trends in Cognitive Sciences, 1422):44-1155 February 2024. ISSN 1364-6613. doi: 10.303 016.
Sheng Xu and Guiliang Liu. Robust inverse constrained reinforcement learning under model misspecification. In Proceedings of the 41st International Conference on Machine Learning. ICML. JoLR.org, 2024.
Ruoyu Xue, Jingyi Xu, Sounak Mondal, Hieu Le, Greg Zelinsky, Minh Hoai, and Dimitris Samaras. Few-shot personalized scanpath prediction. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13497-13507, 2025.
Zhibo Yang, Lihan Huang, Yupei Chen, Zijun Wei, Scoyoung Ahn, Dimitris Samaras, Gregory Zelinsky, and Minh Hoai. Predicting goal-directed human attention using inverse reinforcement learning. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
Brian D. Ziebart, Andrew Maas, J. Andrew Bagnell, and Anind K. Dey. Maximum entropy inverse reinforcement learning. In Proceedings of the 23rd National Conference on Artificial Intelligence Volume 3, AAAI088, page 1433-1438, AAAI Press, 2008. ISBN 9781577353683.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations are discussed in the 'Limitations and future work' Section of the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
Theories and necessary proofs are available in Section 3 of the the manuscript.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The authors performed experiments in two different environments validate the proposed theory: a simple gridworld environment, and a high-fidelity driving simulator. The necessary information for reproducing the gridworld experiments are provided in Sections 1. and 4.2 and Appendix A while additional information for the simulator-bast experiments are provided in Section 4.3 and Appendix B
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
cases, in which
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //nips. fliaps.cccpublice wwtppbbiccgguueeeeeoooooocppcped
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details for the gridworlds experiment only require information about gridworld configurations, which are provided in Appendix [A] Information about hyperparameters for the simulator-based experiment is provided in Section 4.3.2 and scenes that were used for the experiment are illustrated in Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
The gridworlds paradigm is a standard paradigm that can run on almost any modern Personal Computer with relative ease. The resources used to the simulator have been mentioned at the end of Section 4.3.1 as well as Appendix F More information about the requirements for the simulator is available in the cited material.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: The positive impacts of our proposed theory on machine teaching are discussed in the introduction.
18

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
No such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The GPUDrive simulator, PPO model trained on the simulator, and the Waymo Open Motion dataset were used for the second set of experiments. All three credited Section
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
19

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, palasets.papersstt paperswi thcode. com con/dataset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Human data was not collected
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Human data was not collected
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: LLMs used
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer L.M policy hatea://maarrppppecc///feereeees for what should or should not be described.
21

---

A Estimating heuristics in tabular setting
grid-O
grid-1
grid-2
grid-3
grid-4
grid-5
grid-6
grid-7
grid-8
grid-9
grid-10
grid-11
grid-12
grid-13
grid-14
0
0
grid-15
grid-16
grid-17
grid-18
grid-19
8
grid-20
grid-
grid-22
grid-23
grid-24
Figure 6: Driving World scenarios used for estimating heuristics in tabular setting. The true reward function and dynamics are the same as those described in Figure []
22

---

B Waymo Open Dataset scenes selected for our experiment
LT3
Figure 7: Trajectory illustrations for the 10 scenes chosen from the Waymo dataset. Trajectories marked in dark red represent the movement the ego vehicle logged in the original dataset. Other colors represent the movements of other vehicles in the scene.
23

---

C GPUDrive Heuristic Biases
C.1 Deviation from Ego Heading bias
The DfEH bias (ōDyEH) computed by taking the average angle (in radians) between the direction of heading of the ego-vehicle and the positional direction of other vehicles (relative to the ego) in the construal.
Positional Direction
Deviation
Ego Heading
Figure 8: Visual representation for the calculation of Deviation from Ego Heading bias {ōnFEN).
NOTE: Car cliparts obtained from freepik
C.2 Relative Heading bias
The RH bias {ΦRн} } was computed by taking the average angle (in radians) between the direction of heading of the ego-vehicle and all other vehicles in the construal.
Other Heading
Ego Heading
Deviation
Figure 9: Visual representation for the calculation of Relative Heading bias (ΔnH). NOTE: Car cliparts obtained from freepik
24

---

C.3 Deviation from Ego Collision bias
The DIEC bias feDjEC} was computed by taking the average value of the dot product of the relative velocity and the relative displacement of each vehicle in the construal, relative to the ego:
1 i=1 ΦDJEC N (V.- (V vo) -PPo; Pose)
E
Where, N is the number of vehicles in construal; Vú and Ve represent the velocities of the current vehicle and the ego vehicle, respectively; and Pos; and Pos, represent the positions of the current and ego vehicles, respectively. Values for this bias were normalized to [-1, 1J. Where 1 indicates that vehicles in the construal are heading towards the ego vehicle, while a value -1 indicates that vehicles moving further away from the ego.
D Algorithm: behavioral utility
The algorithm block summarizes our approach for computing the behavioral utility (VOR) of each construal each scene. The construed model policy лe obtained by masking the state-space of the optimal baseline model.
Algorithm Behavioral Utility Calculation
for scene all_scenes do
So(scene)
Start state for the scene b Loop through list of construals given
for in construals(s) do
N t 40
construal aeddddeeardssss +- empty_listt
Tc + xCC)
Get construed model policy
while N do
reward < simulate(s, Tc)
construal wtw.aaarraarrrsdssrrrreeeaaa
N - N
end while eeehaviorol_uiiiiieesseeeee C) Average
end for
end for
25

---

E Algorithm: A inference
The algorithm block describes our approach for inference of heuristic biases (Aş). The estimation is performed by maximizing the probability of observing set of sampled trajectories over values of λи. The maximization in our case was performed using a Bayesian optimization technique for GPUDrive, and gradient-based optimizer for the DrivingWone World paradigm.
Algorithm 2 Inference Logic
procedure Get_Ptrajj (rat_Prrjsssssspplee (sampled_ trajs, Aarg)
Aorglist + emplyy_iss()
for ( in sampled_traji do
[_lisl &- emply_list)
s So(C)
Start state for trajectory
for C' in construals(s) do
TC + TC))
Get construed model policy
Paic t (s (a( 8t)
Poparra +- evon(s,O)+ (aan (s.C)
inist.insert Paice * Patpara)
end for
Pepdarg + Eciil
weg.iist.inerr (PcIdar)) end for P Damplel/ara return P ssamplelAarg end procedure
f IIragg list
sampled trajs + simulate(TAxrr..)
Sample trajectories for agent with Arrue heuristics Randomly initialize Estimate
Ainit t somple((>DEEn Ann ADfECH)
Aast f- marimizez Get_Ptrajj (traa/A (sampled_ trajs, Ainit)
Time Complexity
In this section, we report our algorithm execution time our experiments with GPUDrive. This because the data generated by the simulator is good approximation of real-world high-dimensional complex data.
We had to overcome several computational bottlenecks to perform inference on real-world Waymo data using GPUDrive. The complete pipeline involves: (1) computing an optimal policy, (2) comput- ing behavioral utilities of construals by simulating the optimal policy is construed observation spaces, and (3) performing maximum likelihood estimation over continuous multi-dimensional lambda values.
One of the algorithmic contributions of paper is policy, which essentially involves amortizing the cost by pre-training a generalist policy using standard RL methods. This be done standard GPU within day. GPUDrive includes pre-trained generalist policy that we used for our experiments.
efficient way to compute the optimal
Computation of behavioral utilities requires simulating masked policy on the true dynamics. It took an hour to evaluate 166 construals across 10 Waymo scenarios by sampling 40 trajectories per construal, on Ubuntu High Performance Computing cluster with single RTX8000 GPU and 40GB of RAM allocated to the process. We were only able to partially parallelize the evaluation process due to limitations in the current implementation of the Python interface, but note that evaluation can be significantly sped up by optimally parallelizing the process.
laptop computer (with Intel Core Ultra 7 165H CPU and 32GB RAM). It took about min and 30 sec to perform inference over eighty 9-second trajectories (roughly 12 minutes of driving data) for single set of parameters. Our
Finally, we performed inference over heuristic parameters
26

---

current implementation of the inference algorithm is sequential CPU-based process, which can be parallelized and executed on GPUs.
G
Sample Efficiency
To test the sample efficiency of our approach in GPUDrive, evaluated the performance of our algorithm on a range of dataset sizes (expressed as durations of observed real-world driving data), from ten 9-second trajectories (the equivalent of 1.5 minutes) to five hundred 9-second trajectories (75 minutes). Evaluation of performance for different durations was measured over 30 randomly sampled lambda values. Our findings from this experiment reported in Figure 10
AAIP Sample Efficiency
60
Squared Enar
30
20
10
20 30
50
60
70
Observation Data (minutes)
Figure 10: Plot for the efficiency of the AAIP algorithm in estimating true values of lambda with different sample sizes of observation data. The x-axis represents the amount of observation data (in minutes) used for the inference process, while the y-axis represents the mean (and standard error) squared error between true and estimated lambdas. The trend suggests that the algorithm converges around 12 minutes of observed driving (eighty trajectories).
27


Reviews:
[{"review_id": "j1vBYf68ln", "reviewer": "Reviewer_tfig", "summary": "This paper highlights the impact of human cognitive biases and introduces attention-aware inverse planning, a method for inferring attentional biases from human behavior. It distinguishes this approach from standard inverse reinforcement learning and presents an algorithm that integrates deep reinforcement learning with cognitive modeling. Applied to driving data from the Waymo Open Dataset, the method uncovers drivers’ attentional strategies.\n\nThis paper makes a meaningful contribution to understanding and modeling human cognitive biases in decision-making. Despite some limitations in representation and modeling complexity, the framework is well-motivated, novel, and reasonably validated. I recommend acceptance.", "questions": "+ Could attentional biases be directly incorporated into the inverse reinforcement learning framework? For example, [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning, Zhang et al - CVPR 2020] model goal-directed attention using IRL.\n\n+ Is the construal mechanism conceptually related to personalized saliency or scanpath prediction? For instance, Xue et al. (Few-shot Personalized Scanpath Prediction - CVPR 2025) propose a method for few-shot personalized scanpath prediction, claiming the ability to learn individualized scanpaths with only a few episodes.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n+ Clarity and Motivation: The paper is well-written and clearly motivated, with a strong theoretical foundation and relevant citations.\n\n+ Novelty and Relevance: The focus on cognitive biases and limited attention in human decision-making is timely and important.\n\n+ Framework Design: The proposed framework is conceptually simple yet general, combining reinforcement learning components (reward, value, policy) with heuristics-based modeling of attentional biases.\n\n+ Scalability: Application to large-scale driving data showcases the method’s practical viability and relevance to real-world problems.\n\nWeaknesses:\n\n+ Simplified Construal Representation: The construal state is modeled as a set of objects, without accounting for inter-object relationships or spatial-temporal dynamics, which may limit expressiveness.\n\n+ Pipeline Complexity: The framework is not end-to-end; it relies on object detection as a preprocessing step to construct the construal space. This may introduce dependencies and limit generalizability.\n\n+ Heuristic Modeling: the heuristics used to uncover the cognitive biases are relatively simple and may not capture richer cognitive phenomena such as relational attention or dynamic prioritization.", "was_revised": true, "final_justification": "I have read the rebuttal and found it addresses my concerns quite well. I recommend accepting this paper.", "rebuttal": "We are grateful to the reviewer for their interest in our work and their thoughtful comments. Below we address the main questions that were brought up and respond to some of the weaknesses noted:\n\n\n> Q1 Could attentional biases be directly incorporated into the inverse reinforcement learning framework? For example, [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning, Zhang et al - CVPR 2020] model goal-directed attention using IRL.\n\nWe thank the reviewer for pointing us to the paper by Zhang et al (2020), which models scanpaths (i.e., gaze patterns) as goal-directed by learning an eye fixation policy with IRL. The types of tasks that they applied their method to are ones in which people are shown an image and must find a target object (e.g., identify a fork in a scene of a dinner table). Identifying certain objects in a scene is an important subtask in more complex tasks such as driving (e.g., to decide if you can cross an intersection, you need to identify the streetlight, among other things). Thus, the attentional biases learned by Zhang et al’s IRL method could be used to extract heuristics that could be used in our AAIP method (or a method that combines IRL with AAIP). This is an interesting direction that builds on the previous results in the scanpath literature that we were previously unaware of. In the working manuscript, we now reference Zhang et al.’s paper as well as discuss how such methods could be used in concert with our approach.\n\n> Q2: Is the construal mechanism conceptually related to personalized saliency or scanpath prediction? For instance, Xue et al. (Few-shot Personalized Scanpath Prediction - CVPR 2025) propose a method for few-shot personalized scanpath prediction, claiming the ability to learn individualized scanpaths with only a few episodes.\n\nWe appreciate the reviewer raising this connection to Xue et al. (2025). Xue et al develop a novel algorithm for learning individualized scanpaths (gaze patterns) efficiently from subjects who are freeviewing or searching images. The construal mechanism overlaps with this approach to gaze modeling in that both are models of attention. However, a key difference is that the construal mechanism provides an account of how attention is affected by the need to plan—it models what someone would pay attention to if they needed to plan a sequence of actions in a certain situation. Xue et al. show how you can efficiently learn personalized scanpaths in a setting that does not involve planning, but their general approach could provide a template for also modeling construals. We see two main challenges to applying Xue et al.’s particular method to the problem of value-guided construal: (1) incorporating a notion of planning into their architecture, and (2) modeling attention and/or gaze as an unobserved variable that is marginalized over since in our setting we do not assume access to gaze data. As with the Zhang et al. paper, this is an interesting connection to the existing literature that we had not considered and now reference in both the Background and future directions.\n\n> Simplified Construal Representation: The construal state is modeled as a set of objects, without accounting for inter-object relationships or spatial-temporal dynamics, which may limit expressiveness.\n\nThis comment as well as R3’s point about addable state representations highlights an important feature of our model that we did not make clear in the original submission. We apologize for this confusion and hope to clarify it here. \n\nAlthough the construal state $s[c]$ is modeled as a set of objects, the plan computed with respect to $s[c]$ **takes the inter-object relationships and spatio-temporal dynamics into account through the transition function**. For example, suppose we have a state with objects {$o_1, o_2, o_3, o_4, o_5$}, but our construal is $c =$  {$o_1, o_2, o_3$}. The optimal policy $\\pi_{c}(a \\mid s)$ is in fact being computed wrt dynamics that incorporate all the interactions between $o_1$, $o_2$, and $o_3$ while ignoring anything having to do with $o_4$ or $o_5$.\n\nThe confusion comes from the fact that in section 3.2, we jumped to the generalist policy formulation without elaborating on the more basic formulation. In the original value-guided construal formulation, an optimal policy for a construal $c$ is associated with a value function with respect to a construed reward function and task, i.e.:\n\n$Q_{c}(s, a) = R_{c}(s, a) + \\gamma\\sum_{s’} T_{c}(s’ \\mid s, a) V_c(s’)$\n\nThe reward function $R_c$ and transition function $T_{c}$ incorporate the interactions between objects in $c$ (and nothing else). This means the value functions and policy (i.e., $V_c$, $Q_c$, and $\\pi_c$) also incorporate information about interactions. In the generalist policy formulation, rather than express separate $R_c$ and $T_c$ for each construal $c$, we assume we have $R$ and $T$ that can operate over masked states, $s[c]$, such that $R_c(s, a) = R(s[c], a)$ and $T_c(s’ \\mid s, a) = T(s’[c] \\mid s[c], a)$. This assumption on $R$ and $T$ makes things notationally convenient (fewer subscripts) and also allows the use of a pre-trained generalist policy described later in the paper. We remarked upon this assumption in section 3.1 but we did not make the connection explicit. \n\nIn our working manuscript we have added these details to clarify that the construal state does indeed incorporate inter-object interactions. \n\n> Pipeline Complexity: The framework is not end-to-end; it relies on object detection as a preprocessing step to construct the construal space. This may introduce dependencies and limit generalizability.\n\nThe reviewer is correct that the current method is not end-to-end and currently requires object detection as a preprocessing step. However, we would like to note that this approach is widely used in work on autonomous driving as well as human-machine interaction (e.g., semi-automated driving, human-machine teaming, machine teaching). For example, perception stacks in autonomous driving typically include object detection (https://arxiv.org/abs/2106.12735) and tracking (https://arxiv.org/abs/2101.02702). Additionally, this approach is useful since it allows us to work in settings with sparser data and provides more control over the internal representations the algorithm uses.\n\nIn the case of human-machine interaction, we especially care about having algorithms that produce interpretable representations and also for those algorithms to accurately reason about people’s internal representations. People reason about the world in terms of objects and their interactions (“Functional Significance of Visuospatial Representations”, Tversky, 2005), and to learn similar kinds of representations, an end-to-end system would need much more data than is typically available in human interaction settings. Moreover, even if such data were available, there is unfortunately no guarantee that an end-to-end system would form similar representations to people, which is often important for interactive systems that reason about human decisions. That said, we view our approach as compatible with more data-driven approaches and are currently exploring extensions in which components like the attentional heuristics that are currently presented to the model can be learned from data.\n\n> Heuristic Modeling: the heuristics used to uncover the cognitive biases are relatively simple and may not capture richer cognitive phenomena such as relational attention or dynamic prioritization.\n\nWe agree with the reviewer that more complex heuristics would be an exciting direction for future work and include a discussion of this in the working manuscript. Here, we address some of the reviewer’s concerns about psychological phenomena explicitly mentioned: (1) dynamic prioritization could be implemented as a bottom-up process where changes in object saliency, or significant changes in ego state could trigger an attention re-allocation event. (2) relational attention may be incorporated by deconstructing larger multi-object construals, where values are computed individually for sub-construals that reflect different heuristic preferences, limiting relational dynamics of objects within each sub-construal. \n\nAdditionally in our working manuscript, we explore two new complex heuristics (relative heading and collision trajectory), in follow up experiments with GPUDrive. We have results for joint inference of the three continuous space heuristics, where we show excellent recoverability of the heuristic parameters (R-squared: 0.77, 0.76, 0.83)."}, {"review_id": "c4hMZNvTYH", "reviewer": "Reviewer_3HcF", "summary": "This paper introduces attention-aware inverse planning, aiming to infer a person's attentional biases from their behavior. Unlike standard IRL, which assumes near-optimal decision-making, this work models how limited attention shapes choices using a parameterized bias function grounded in the value-guided construal framework. The authors also propose a scalable algorithm combined with deep RL and cognitive modeling to learn these biases via maximum likelihood. Experiments on show that this method outperforms IRL in capturing attention-limited behaviors in simulation experiments.", "questions": "What is the latency of the proposed method?", "limitations": "See weakness.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I like the motivation of this work. Specifically, the authors address the gap between cognitive science findings and standard ML models. This is interesting to me especially about the realistic modeling framework grounded in human cognition. Besides, the proposed method provides a systematic, extensible approach for explaining, predicting, and influencing deviations from ideal rational behavior. Experiments also demonstrate the effectiveness of the proposed methods with outperforming standard IRL. \n\nHowever, the attentional heuristics used are relatively simple. The authors also mentioned this. Capturing more nuanced real-world biases may require richer heuristic models. Is there some better ways? Besides, although the method is claimed to be scalable and is demonstrated on the Waymo Open Dataset, the inference process—particularly the marginalization over all possible construals (as in Eq. 6)—can be computationally expensive in theory, raising concerns about scalability to more complex domains. It would be better for the authors to provide some latency analysis.", "was_revised": true, "final_justification": "The authors addressed my concerns, I recommend accept.", "rebuttal": "We thank the reviewer for their interest in our work!\n\n> Q1 What is the latency of the proposed method?\n\nOur current implementation performs inference in Python on a CPU since this gives us the fine-grained control of the various components. Further, inference over various construals are currently performed sequentially; processing time can be reduced significantly through parallelization. Even using this non-optimized version, we are able to perform inference (estimate heuristic parameters for a trajectory of actions) for the equivalent of 12 minutes of driving data in 3 minutes and 30 seconds. \n\nAdditionally, since our original submission, we have performed more systematic analysis of the performance of our algorithm in terms of its real-time data complexity. The updated algorithm can infer multiple continuous heuristic parameters jointly using gaussian process optimization (the original manuscript reported an experiment with only a single parameter naively optimized over discrete values). In the working manuscript, we now report results for estimating 3 continuous heuristic parameters. To evaluate real-time data efficiency on the Waymo scenarios in the GPUDrive simulator, we evaluated the algorithm with different amounts of driving data, ranging from the equivalent of 1.5 minutes of driving to 75 minutes of driving. With 1.5 minutes of driving data, the mean squared error between true and estimated parameters is 52.64 (averaged over 30 simulations). At 12 minutes of driving data this decreases to 19.56 and stays around this value with more data. A plot of these results is now included in the supplement of the manuscript (we did not include it here due to the rebuttal rules)."}, {"review_id": "Q0Ld9YHzlZ", "reviewer": "Reviewer_MaQm", "summary": "This manuscript introduces a framework called Attention-Aware Inverse Planning (AAIP) to interpret human cognitive limitations based on their behaviors. The core assumption is that people can’t plan with all the sensory inputs they receive from the external world due to perceptual costs, and thus they plan with a limited subset of objects in that state space. Unlike Inverse Reinforcement Learning (IRL), the framework does not assume a special form of reward function for these sub-state decision makers. The model aims to work backward from an observed behavior, such as a driving trajectory, to estimate the specific parameters of cognitive biases that have caused such limitations. The authors test this approach using simulated agents in a simple grid world and a more realistic driving simulator, showing that their method can successfully recover the built-in biases, unlike IRL.", "questions": "1. In lines 214-215, the author used a “known” reward function for IRL, and with this reward design, the AAIP outperforms IRL in later experiments. However, I wonder if we can sample a reward function from more expressive reward function spaces; will things be different? Or, to put it more generally, I wonder if the deficit of IRL, which assumes full observation of states, can always be compensated by reward function design. Is there any bridge that can unify these two frameworks? Or is there an impassable divide between them? For example, can we borrow ideas from Partially Observable MDP to address the problem of vanilla IRL? These questions go far beyond the scope of the current paper, but since one of the core experiments in this study involves comparing two frameworks, I think a thorough discussion with some quantitative analysis would make the results far more convincing.\n\n2. Similar issues like Q1 regarding the specific form of cognitive costs and heuristic feature engineering exist in the manuscript. I expect more discussion on this in the final sections.", "limitations": "1. The study relies entirely on synthetic data and lacks validation with actual human behavior. The authors allude to this as a future step but don't frame it as a direct limitation of their current work.\n2. The framework cannot discover new cognitive biases; they must be defined by researchers beforehand. This makes it implausible in end-to-end cases where biases must be inferred automatically. Instead, the authors claim that more “complex heuristics” will be used in those cases, which I don’t see as a good choice for real-world problems.\n3. The model's focus is narrowly restricted to sub-optimality from inattention, ignoring other key factors in human decision-making, such as risk aversion/preference or memory constraints. The authors do not discuss this limitation.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1.  AAIP is  a new and important way to model behavior by inferring cognitive biases, moving beyond the limitations of traditional IRL, which might be crucial for understanding human behaviors and further design machines that interact with humans.\n2. The model is well-grounded in established cognitive theories like bounded rationality and value-guided construal, giving it strong theoretical validity and explanatory power.\n\n**Weaknesses**\n\n1. The paper's primary weakness is its exclusive reliance on synthetic data. Without real human data that includes humans' cognitive biases through self-report or eye movement, it’s hard to verify the explanatory power of the framework.\n2. The equation used to represent the value function assumes an addable state representation, which is not true in many real-world settings because the actual value in a task may rely on more than two-order level relationships of objects. This is also a core issue in modern DL interpretability studies where the inputs are high-dimensional, like images and videos. Therefore, I doubt if this framework can be applied to problems that are solved in an end-to-end fashion.\n3. The framework makes very simplified assumptions about decision makers’ attention, that they are simply selecting a fixed group of objects in the whole episode. This is a significant simplification that doesn't reflect the dynamic nature of attention in most real-world situations.", "was_revised": true, "final_justification": "I have read the rebuttal and found it partially addresses my concerns. I recommend accepting this paper.", "rebuttal": "We thank the reviewer for their interest in our work and their helpful comments. Below we first respond to the questions and then elaborate on additional points raised.\n\n> Q1 In lines 214-215, the author used a “known” reward function for IRL, and with this reward design, the AAIP outperforms IRL in later experiments. However, I wonder if we can sample a reward function from more expressive reward function spaces; will things be different? Or, to put it more generally, I wonder if the deficit of IRL, which assumes full observation of states, can always be compensated by reward function design. Is there any bridge that can unify these two frameworks? Or is there an impassable divide between them? For example, can we borrow ideas from Partially Observable MDP to address the problem of vanilla IRL? These questions go far beyond the scope of the current paper, but since one of the core experiments in this study involves comparing two frameworks, I think a thorough discussion with some quantitative analysis would make the results far more convincing.\n\nWe thank the reviewer for raising these important questions about the relationship between IRL and AAIP that get to the heart of our work. The simulations in section 3.5 were designed as a simple quantitative case study to show some of the limits of IRL, but we completely agree that an exciting direction for future work is a more formal analysis of these two paradigms. \n\nIn the discussion, we now highlight work that we believe sheds light on the relationship between IRL and AAIP. Specifically, theoretical results in “Robust IRL” show that if the transition function of an IRL learner is misspecified, this affects the bound on the gap between the value of the learned policy and the true policy (https://arxiv.org/pdf/2007.01174). Applying IRL to an attention-limited decision-maker’s actions is similar in that such a decision-maker is acting wrt a simplified transition model. Thus, we know in principle that there can be a recoverability gap when doing IRL in general. It is an open question how pervasive this gap is in real-world driving scenarios or other tasks, but the simple example in section 3.5 suggests it is not difficult to construct such scenarios.\n\n> Q2 Similar issues like Q1 regarding the specific form of cognitive costs and heuristic feature engineering exist in the manuscript. I expect more discussion on this in the final sections.\n\nWe agree that it is important to extend this work to learning cognitive costs and heuristics from data. In the discussion of the manuscript, we now include the following sentence: “Similar to how we can infer the heuristic $\\lambda$ weights from data, we could also infer parameters of the heuristic feature functions themselves $\\{\\phi\\}$ from data from the gradient of the likelihood, though this would be more data-intensive. Future work should explore this possibility to extend the generality of our proposed framework.”\n\n> The equation used to represent the value function assumes an addable state representation, which is not true in many real-world settings because the actual value in a task may rely on more than two-order level relationships of objects.\n\nThis is an important comment that echoes a point made by R1. **We did not make clear in the original submission that the value functions computed by the model does take into account higher-order object relationships**. We include our explanation here for the reviewer’s convenience:\n\nAlthough the construal state $s[c]$ is modeled as a set of objects, the plan computed with respect to $s[c]$ **takes the inter-object relationships and spatio-temporal dynamics into account through the transition function**. For example, suppose we have a state with objects {$o_1, o_2, o_3, o_4, o_5$}, but our construal is $c =$  {$o_1, o_2, o_3$}. The optimal policy $\\pi_{c}(a \\mid s)$ is in fact being computed wrt dynamics that incorporate all the interactions between $o_1$, $o_2$, and $o_3$ while ignoring anything having to do with $o_4$ or $o_5$.\n\nThe confusion comes from the fact that in section 3.2, we jumped to the generalist policy formulation without elaborating on the more basic formulation. In the original value-guided construal formulation, an optimal policy for a construal $c$ is associated with a value function with respect to a construed reward function and task, i.e.:\n\n$Q_{c}(s, a) = R_{c}(s, a) + \\gamma\\sum_{s’} T_{c}(s’ \\mid s, a) V_c(s’)$\n\nThe reward function $R_c$ and transition function $T_{c}$ incorporate the interactions between objects in $c$ (and nothing else). This means the value functions and policy (i.e., $V_c$, $Q_c$, and $\\pi_c$) also incorporate information about interactions. In the generalist policy formulation, rather than express separate $R_c$ and $T_c$ for each construal $c$, we assume we have $R$ and $T$ that can operate over masked states, $s[c]$, such that $R_c(s, a) = R(s[c], a)$ and $T_c(s’ \\mid s, a) = T(s’[c] \\mid s[c], a)$. This assumption on $R$ and $T$ makes things notationally convenient (fewer subscripts) and also allows the use of a pre-trained generalist policy described later in the paper. We remarked upon this assumption in section 3.1 but we did not make the connection explicit. \n\nIn our working manuscript we have added these details to clarify that the construal state does indeed incorporate inter-object interactions. \n\n> The framework makes very simplified assumptions [...] that doesn't reflect the dynamic nature of attention in most real-world situations.\n\nWe agree and acknowledge that this is an important next step, and consequently, address this limitation in the discussion as a problem for future work. However, we also want to point out that the degree to which this assumption is problematic depends on the timescale of trajectories. Individual trajectories in the Waymo dataset represent 9 seconds of real-time behavior, which could limit how much dynamic attention is occurring. Additionally, in previous psychology work on value-guided construals done with maze navigation, it was found that a single fixed construal approximated the “cumulative construal” when attentional processes were modeled dynamically (Ho et al., 2022; supplemental material). In other words, the set of things one paid attention to over the course of an entire trial was similar to the set of things one would pay attention to all at once, if they had to. While this still results in a loss of information (i.e., precisely when someone paid attention to something), it shows that assuming a single fixed construal is a reasonable simplification for short timescales (~5-10 seconds). Acknowledging the caveat that these were experiments in mazes and not in a driving domain, this suggests that assuming that drivers select a fixed group of objects across a 9 second episode is a reasonable simplification.\n\n> The study relies entirely on synthetic data and lacks validation with actual human behavior. The authors allude to this as a future step but don't frame it as a direct limitation of their current work.\n\nWe thank the reviewer for pointing out this oversight and now explicitly state this limitation of the current work. In the discussion we have now added “Finally, a key limitation of the current work is that it focuses on validation on simulated data and not human behavior.”\n\n> The framework cannot discover new cognitive biases; they must be defined by researchers beforehand. This makes it implausible in end-to-end cases where biases must be inferred automatically. Instead, the authors claim that more “complex heuristics” will be used in those cases, which I don’t see as a good choice for real-world problems.\n\nWe agree with the reviewer’s comment that the current paper does not discover new biases. Part of the reason for this is that we would like our framework to allow users to evaluate pre-specified or hypothesized biases, which is useful in cognitive science applications as well as other domains where interpretability is important. That said, we believe that our framework can straightforwardly accommodate inferring new attentional biases by allowing the $\\phi$ functions (described in section 3.3) to be parameterizable and learned from data with gradient-based methods. The main challenge with doing this is that it would require larger data sets, but we see our approach as compatible with data driven methods and see this as an exciting direction for future work.\n\nAdditionally, in relation to the comment about end-to-end learning, we would like to clarify that while our approach does require pre-processing (e.g., extracting object representations), we take this approach intentionally since it allows us to work in settings where data is less abundant or the internal representations used by an algorithm are important. This includes work on autonomous driving as well as human-machine interaction (e.g., semi-automated driving, human-machine teaming, machine teaching).\n\n> The model's focus is narrowly restricted to sub-optimality from inattention, ignoring other key factors in human decision-making, such as risk aversion/preference or memory constraints. The authors do not discuss this limitation.\n\nWe agree with the reviewer that the current paper focuses on sub-optimality from attention and does not incorporate other kinds of biases. Our framework can be extended to incorporate these additional factors, as we discuss in the new limitation section (pasted below):\n\nAlthough we focus on one cognitive dimension of suboptimal driving behavior, other psychological factors, such as propensity for risky taking, ability of risk perception, and slow response times, can also lead to suboptimal decisions in driving tasks. It is possible to incorporate other psychological factors into our model in the future. For instance, joint inference of heuristic biases and reward functions would allow for simultaneously estimating a driver's underlying heuristics and tendency for risky behavior."}, {"review_id": "hKWrLM7lRZ", "reviewer": "Reviewer_qcxq", "summary": "This paper introduces attention-aware inverse planning (AAIP), a framework for estimating people’s task-specific attentional bias from observed behavior by introducing an inverse problem of the value-guided construal model from previous work [1]. The authors (1) define the inverse problem of inferring bias parameters from state-action trajectories, (2) provide an MLE-based inference algorithm for this problem, (3) validate it in a tabular gridworld with accurate biases recovery performance, and show standard inverse reinforcement learning (IRL) fails to capture attention-limited policies, and (4) scale up to driving scenes simulated from the Waymo Open Dataset using a GPUDrive simulator and show accurate bias parameters recovery.\n\n[1] Ho et al., People construct simplified mental representations to plan, Nature (2022)", "questions": "1. Given that AAIP currently assumes a known reward function, how might the framework be extended to infer both the reward and the attentional‐bias parameters λ when the reward is unknown or only partially specified?\n2. Could you provide the time complexity and empirical runtime of AAIP?\n3. Could you elaborate on why error bars and significance tests are unnecessary for experiments in this paper?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1. This paper clearly defines the attention-aware inverse planning problem and explains the motivation behind proposing this new problem (human behavior often has attention-related bias that cannot be captured by the standard IRL model). The problem proposed can be viewed as an inverse problem of bias parameters in the value-guided construal model from previous work [1].\n2. This paper conducted experiments in both tabular and scalable setups, showcasing the accurate biases recovering performance of the proposed algorithm. In the tabular case, it also shows that standard IRL fails to capture attention-limited policies, emphasizing the need of the proposed AAIP framework to accurately model agent behavior. \n\n**Weaknesses**\n1. The AAIP framework treats the reward as a fixed, known input rather than inferring it from data. In contrast, standard IRL recovers both reward and policy. Relying on a predefined reward limits AAIP's applicability in complex settings where the true reward structure is unknown or difficult to specify.\n2. The identifiability of bias in AAIP remains unclear. While the paper proposes a maximum‐likelihood estimator, it offers no analysis on when and to what degree the bias weights can be uniquely recovered from behavior. Empirically, both experiments explore only very simple bias spaces, so it is unknown how inference would perform in more complex settings. Even in the scalable Waymo experiment, the learnable bias parameter λ lies on a small discrete 1D space.\n3. There is no discussion of the algorithm’s time complexity, nor empirical runtime report. Such discussion would help clarify computing resource requirements and efficiency.\n4. No statistical significance info is provided for experiments. While authors argue that error bars and significance tests are unnecessary for this paper in the checklist, IRL literature typically reports results of multiple runs with different random seeds. Including such statistics would help readers understand the robustness of the reported bias‐recovery performance.\n5. Although the inference method is described in the text, there is no pseudocode or algorithm block to represent the overall inference procedure. A clear, step‐by‐step outline, which specifies inputs, outputs, and optimization steps, would make the procedure easier to understand. \n\nOverall, this paper introduces an interesting inverse problem of value-guided construal models that has potential in cognitive science research. However, I feel it is not yet ready for publication in its current form, given the limited scope of experiments and theoretical analysis.", "was_revised": true, "final_justification": "Since the authors have adequately addressed my concerns over extending AAIP to inferring reward function and empirical runtime, and (claimed to) have performed additional experiments regarding statistical significance & bias recovery performance and revised the manuscript accordingly, I increase my score to 4 (Borderline Accept).", "rebuttal": "We thank the reviewer for their interest in our work and their thoughtful comments. We have thoroughly considered their feedback about the limitations of our work and have attempted to address them in the working version of the manuscript. Below we respond to the questions and elaborate on additional points raised.\n\n> Q1 Given that AAIP currently assumes a known reward function, how might the framework be extended to infer both the reward and the attentional‐bias parameters λ when the reward is unknown or only partially specified?\n\n> [related comment] The AAIP framework treats the reward as a fixed, known input rather than inferring it from data. In contrast, standard IRL recovers both reward and policy. Relying on a predefined reward limits AAIP's applicability in complex settings where the true reward structure is unknown or difficult to specify.\n\nWe thank the reviewer for this question (and related comment). While we believe that AAIP is an interesting problem even when rewards are known, we entirely agree that inferring rewards is an important extension of our current approach. Indeed, it can be done; Rane et al. (2023) demonstrated that joint Bayesian inference of rewards and construals better capture human judgments than simply estimating rewards from behavior using IRL models. Note though that that work differs from ours in several ways—it was done with simple gridworlds, focused on comparison with human judgments, and did not use *value-guided* construals.\n\nIn the context of the current work, we can sketch out how joint inference over heuristic parameters and rewards should be specified. Given observed behavior $\\zeta = \\langle s_1, a_1, …, s_T, a_T \\rangle$, the posterior over reward function parameters $\\theta$ in Bayesian IRL is:\n\n$$\nP(\\theta \\mid \\zeta) \\propto P(\\theta) \\prod_{t} \\pi^*_\\theta(a_t \\mid s_t)\n$$\n\nwhere $\\pi^*_\\theta$ is the optimal policy in the task under a reward parameterization $\\theta$. Jointly inferring rewards and attention parameters $\\lambda$ would have the following posterior:\n\n$$\nP(\\theta, \\lambda \\mid \\zeta) \\propto P(\\theta)P(\\lambda) \\sum_c \\pi_{\\lambda}(c \\mid s) \\prod_t \\pi^*_{\\theta}(a_t \\mid s_t[c])\n$$\n\nWhere $\\pi_{\\lambda}(c \\mid s)$ is from Equation 3 in the original submission. Note that this is a more complex inference problem since the construal is a hidden state that persists over the course of the trajectory. Indeed, we speculate that this joint inference can only be robustly solved if treated in a fully Bayesian manner—that is, integrating over possible $\\theta$ and $\\lambda$ and not simply solving for MLE parameters. This is an important direction that we hope will be explored more thoroughly in future work, and in the discussion now write “The current work focuses on AAIP given a known reward, but in many domains rewards are unknown or individuals may have idiosyncratic preferences. Thus, the applicability of AAIP will be greatly expanded by developing new methods that allow for joint inference of attentional heuristics and rewards.”\n\n> Q2 Could you provide the time complexity and empirical runtime of AAIP?\n\n> [related comment] There is no discussion of the algorithm’s time complexity, nor empirical runtime report. Such discussion would help clarify computing resource requirements and efficiency.\n\nWe thank the reviewer for raising this point. \n\nPrior to inference, our algorithm requires evaluating construals to calculate their behavioral utility and therefore their probability of being chosen. This means there are two main computational bottlenecks: (1) computing an optimal policy and (2) evaluating an optimal policy in the true task using a simulator. One of the algorithmic contributions of our paper is a more efficient way to compute the optimal policy, which essentially involves amortizing the cost by pre-training a generalist policy using standard RL methods (e.g., PPO on the GPUDrive platform). This can be done on a standard GPU within a day (note GPUDrive includes a pre-trained generalist policy that we used). The *evaluation stage* requires simulating a masked policy on the true dynamics. This took an hour to evaluate 166 construals across 10 Waymo scenarios by sampling 40 trajectories per construal. We were only able to partially parallelize the evaluation process due to limitations in current implementation of the Python interface, but we note that evaluation can be significantly sped up by optimally parallelizing the process in GPUDrive. At inference time, it takes about 3 min and 30 sec to perform inference over 80 trajectories (roughly 12 minutes of driving data) for a single set of parameters (machine details: Intel Core Ultra 7 165H laptop CPU with 32GB of RAM). We will release all code used in the project once the paper is published and include information about approximate runtime.\n\nIn addition, we now include results related to the real-time data efficiency of an updated version of our algorithm. The updated algorithm can infer multiple continuous heuristic parameters jointly using gaussian process optimization (the original manuscript reported an experiment with only a single parameter naively optimized over discrete values). In the working manuscript, we now report results for estimating 3 continuous heuristic parameters. To evaluate real-time data efficiency on the Waymo scenarios in the GPUDrive simulator, we evaluated the algorithm with different amounts of driving data, ranging from the equivalent of 1.5 minutes of driving to 75 minutes of driving. With 1.5 minutes of driving data, the mean squared error between true and estimated parameters is 52.64 (averaged over 30 simulations). At 12 minutes of driving data this decreases to 19.56 and stays around this value with more data. A plot of these results is now included in the supplement of the manuscript (we did not include it here due to the rebuttal rules).\n\n> Q3 Could you elaborate on why error bars and significance tests are unnecessary for experiments in this paper?\n\n> [related comment] No statistical significance info is provided for experiments. While authors argue that error bars and significance tests are unnecessary for this paper in the checklist, IRL literature typically reports results of multiple runs with different random seeds. Including such statistics would help readers understand the robustness of the reported bias‐recovery performance.\n\nWe did not include error bars or significance tests since we plotted the raw data and the results appeared visually robust, but in our working manuscript we now include error bars and significance tests for old and new analyses. Specifically, since our submission, we have conducted more extensive GPUdrive experiments in which we attempt to jointly estimate 3 heuristics in the scenarios, similar to the DrivingWorld paradigm. For both sets of experiments, we now report significance tests to compute the R-squared value for the model fit (actual vs. estimated heuristic parameters) and p-value measures for both model fits. The P-values for the GPUDrive experiment are in the neighborhood of 1.0E-80. For our gridworld experiment, p-values range around 1.0E-300.\n\n> The identifiability of bias in AAIP remains unclear. While the paper proposes a maximum‐likelihood estimator, it offers no analysis on when and to what degree the bias weights can be uniquely recovered from behavior. Empirically, both experiments explore only very simple bias spaces, so it is unknown how inference would perform in more complex settings. Even in the scalable Waymo experiment, the learnable bias parameter λ lies on a small discrete 1D space.\n\nWe acknowledge more work could be done to analyze the degree to which bias weights can be recovered. In new work, now included in the updated manuscript, we’ve extended the Waymo experiment, where we perform joint inference over a continuous 3D heuristic space. Our results show excellent recoverability with R-squared values 0.77, 0.76, and 0.83 (p < $10^{-4}$) for each of the three complex heuristics even with a modest amount of data (the equivalent of roughly 12 minutes of real-time driving). Further, we discuss specific avenues for moving to larger bias spaces or learning them in our updated discussion.\n\n> Although the inference method is described in the text, there is no pseudocode or algorithm block to represent the overall inference procedure. A clear, step‐by‐step outline, which specifies inputs, outputs, and optimization steps, would make the procedure easier to understand.\n\nWe now include an algorithm block in our paper to lay out our complete inference process."}, {"review_id": "SnDnzXLcuZ", "reviewer": "Reviewer_kdHu", "summary": "This article investigates the interesting topic of how goal-directed actions are influenced by human cognitive biases. Specifically, it examines how limitations in human attention lead to biases in attending to objects, which in turn can significantly affect the actions performed.", "questions": "Please kindly visit the Strengths And Weaknesses. I do not have further questions.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The presentation of the main ideas is clear and consistent.\n- The main research ideas are elaborated well in the introduction and background, and further in Section 3.\n- Experiments are conducted on a synthetic dataset generated using the GPUDrive driving simulator, serving as a proof of concept.\n\n\n**Weaknesses**\n- The paper conducts only limited experiments, all performed on a synthetic dataset. While this serves as a reasonable proof of concept, it leaves unanswered how the proposed ideas would perform in real-world scenarios.\n\n- From what I understand, the experiments use only a single dataset, the Waymo Open Motion dataset, which further limits the work’s generalizability.\n\n- There are no comparisons with related works presented in the paper. The authors state that *“we build on existing work in computational cognitive science,”* but it remains unclear how the proposed model differs from prior work, both in theoretical foundations and applied aspects — for example, in relation to resource-limited planning in humans, such as the value-guided construal model.\n\n- The paper would benefit from a dedicated section for experiments and results. Currently, the presentation of experiments is scattered, making it difficult to follow. I strongly recommend adding a clear Methodology section to describe all aspects of the proposed model, followed by distinct sections for Experiments and Results.\n\n- Additionally, the manuscript lacks a Conclusion section. I suggest including one, even if brief, to summarize key findings and outline future work.\n\n- Figure 2 is difficult to interpret. There does not appear that the caption explaining the figure or how to extract information from it. Additionally, it is unclear how important it is for the reader to understand the estimated heuristic weights in tabular MDPs. Finally, the meaning of the $R^2$ values shown within the figure is not explained.\n\n- Figure 4 is not sufficiently descriptive. The arrows and annotations do not further clarify the context or content depicted, making it difficult to interpret the figure.", "was_revised": true, "final_justification": "Based on the authors’ rebuttal, my main concerns have been addressed, and I have accordingly increased my initial score.", "rebuttal": "We thank the reviewer for their interest in our ideas and their thoughtful critiques. We have attempted to address the weaknesses they note in the working manuscript and believe that this has substantially improved the presentation of our work as well as its limitations.\n\n> The paper conducts only limited experiments, all performed on a synthetic dataset. While this serves as a reasonable proof of concept, it leaves unanswered how the proposed ideas would perform in real-world scenarios.\n\nWe agree with the reviewer that there is still much work to do applying the proposed ideas to real-world scenarios and now note this more explicitly in the paper. To be clear, however, our experiments are not purely synthetic since the Waymo dataset we used are all real-world scenarios. The ego vehicles in those scenarios are controlled by a synthetic PPO model, but they have to contend with other vehicles that follow trajectories logged from the driving behavior of humans. All aspects of the scenarios, apart from the trajectory of the ego vehicle, represent real-world complexity and behavior. We believe this demonstrates that our approach is compatible with the physical constraints and affordances involved in the real-world task of driving.\n\nThat said, related to the concerns of the reviewer regarding real-world applicability, we have also performed a followup study to approximate the real-world data efficiency of our algorithm across data amounts ranging from the equivalent of 1.5 minute to 75 minutes. With 1.5 minutes of driving data, the mean squared error between true and estimated parameters is 52.64 (averaged over 30 simulations). At 12 minutes of driving data this decreases to 19.56 and stays around this value with more data. A plot of these results is now included in the supplement of the manuscript (we did not include it here due to the rebuttal rules). We believe this provides a meaningful demonstration of data efficiency, and it does so precisely because we committed to testing our algorithm in simulations of real-world driving scenarios.\n\n> From what I understand, the experiments use only a single dataset, the Waymo Open Motion dataset, which further limits the work’s generalizability.\n\nWe agree with the reviewer that testing on more datasets and domains would be very valuable. That said, we would like to point out that the Waymo open-motion dataset includes 574 hours of driving-data collected from a wide range of driving contexts and scenarios, and it is a standard dataset, widely used for validation of Machine Learning approaches for driving. For our experiment, we further selected scenarios that involve executing some of the most complex driving maneuvers, which include merging, yielding, and passing other cars. We believe that our algorithm’s ability to reliably estimate heuristics in such complex driving contexts in fact demonstrate the versatility of our approach, at least within the domain of  real-world driving situations.\n\n> The authors state that “we build on existing work in computational cognitive science,” but it remains unclear how the proposed model differs from prior work, both in theoretical foundations and applied aspect.\n\nWe agree with the reviewer that we did not make the difference with prior work clear and appreciate this being pointed out. The current paper is an extension of the previous work by [Ho et al., 2022, 2023], where the authors introduce a framework for modeling limited attention in humans (through value-guided construals). The previous work focused on simple discrete domains (gridworlds) to ensure tight experimental control for human experiments.  The current work extends that framework in three distinct ways: (1) we introduce heuristic construals, (2) we focus on developing a scalable algorithm for the inverse problem, and (3) we validate the approach in the substantially more complex domain of simulated driving in real-world scenarios. Our working manuscript now includes a more explicit discussion of how it differs from prior work in the introduction as well as background sections.\n\n> The paper would benefit from a dedicated section for experiments and results. Currently, the presentation of experiments is scattered, making it difficult to follow.\n\n> The manuscript lacks a Conclusion section. I suggest including one, even if brief, to summarize key findings and outline future work.\n\nWe appreciate the reviewer’s comments on the clarity of the presentation of our work, for both experiments and summarization of the key findings. Our working manuscript has been reorganized. The experiments and results section now contains three experiments. Inference of heuristic parameters in gridworld, comparison with IRL, and Inference of heuristic parameters for GPUDrive. Additionally, a dedicated conclusion section has now been added to the manuscript that summarizes the main contributions of the paper. Our comments about potential future work have been added to a section on “Limitations and Future Work”\n\n> Figure 2 is difficult to interpret. There does not appear that the caption explaining the figure or how to extract information from it. Additionally, it is unclear how important it is for the reader to understand the estimated heuristic weights in tabular MDPs. Finally, the meaning of the values shown within the figure is not explained.\n\nWe have updated the caption for the figure, it now says:\nPlots for model fit results for AAIP: actual weights of heuristic biases plotted against\nmodel estimates, in a simple Gridworld MDP. We simulated the behavior of 1000 agents (with different combinations of weights for three heuristics) by sampling 125 trajectories from 25 different DrivingWorld scenarios for each agent. Then, we attempted to learn the weights from behavior via maximum-likelihood estimation. Each sample represents the true weight of a heuristic for an agent and its estimated value derived from the agent’s behavior. Correspondence between true and estimated weights demonstrates that heuristics that are less consequential (e.g., a bias to ignore cones) are more readily identifiable from behavior (indicated by higher R-squared values). In contrast, biases that could be more consequential (e.g., a bias to ignore parked cars) are less identifiable from behavior because they are outweighed by the effect of value-guided construals. These results illustrate, in a simple setting, the viability and challenges of attention-aware inverse planning.\n\n\n> Figure 4 is not sufficiently descriptive. The arrows and annotations do not further clarify the context or content depicted, making it difficult to interpret the figure.\n\nArrows have been removed, and markers have now been added to draw attention to the various aspects of the figures. In addition, the caption has been updated to:\nThe three figures demonstrate the behavior (driving trajectory) of our PPO agent controlling the ego vehicle under three conditions: (i) when it can see every vehicle in the scene (left); (ii) when it can see a single vehicle that is relevant to the driving plan (middle); and (iii) when it can a single vehicle that is irrelevant to the driving plan (right). The ego trajectories in translucent red represent the behavior of the PPO agent across 40 trials in each condition. Solid lines of all other colors represent the trajectories of other vehicles in the scene. The purple solid lines in the middle and right plots show the trajectory of the single vehicles being observed in each condition. The behavior of our PPO agent (demonstrated by the distribution of ego trajectories) in the optimal condition (left) is similar to the condition where the agent’s construed state space contains a vehicle that is relevant to its plan (middle), as opposed to the condition where it observes a different irrelevant vehicle (right)."}]

Author rebuttal:

Meta-review: The submission proposed an alternative to inverse reinforcement learning (IRL): find the rewards that explain a behaviour. In principle, IRL could be used to capture some forms of bias by agents. The submission proposes to do IRL, but instead of rewards, the bias is towards a subset of objects.

The reviewers agree on the elegance of the idea and the overall presentation.

Among the weaknesses, reviewers thought that focusing on objects might fail to capture relationships, but the authors clarified that point. Another point was that the limited experiments. The authors performed additional experiments.

The authors acknowledge that their method is not end-to-end as it requires object detection, and argue that this is very common in concrete applications and improves interpretability.

The consensus is an explicit acceptance.

Personally, I recommend that the authors clarify the technical similarities between the IRL formulation and the value-guided construal. Reducing aparent differences with IRL might increase the impact of your work.
Decision: Accept (spotlight)