Goal Inference as Inverse Planning

Goal Inference as Inverse Planning Chris L. Baker, Joshua B. Tenenbaum & Rebecca R. Saxe {clbaker,jbt,saxe}@mit.edu Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Abstract many authors (e.g. Nichols and Stich (2003); Baker, Tenen- baum, and Saxe (2006)) have argued that the qualitative de- scriptions of the principle of rationality that have been pro- posed are insufficient to account for the complexities of hu- man goal inference. Further, the qualitative predictions of noncomputational models lack the resolution for fine-grained comparison with people’s judgments. Here, we propose a computational version of this approach to goal inference, in terms of inverse probabilistic planning. It is often said that “vision is inverse graphics”: computational models of visual perception – particularly in the Bayesian tra- dition – often posit a causal physical process of how images are formed from scenes (i.e. “graphics”), and this process must be inverted in perceiving scene structure from images. By analogy, in inverse planning, planning is the process by which intentions cause behavior, and the observer infers an agent’s intentions, given observations of an agent’s behav- ior, by inverting a model of the agent’s planning process. Like much work in computer vision, the inverse planning framework provides a rational analysis (Anderson, 1990) of goal inference. We hypothesize that people’s intuitive the- ory of goal-dependent planning approximates scientific mod- els of human decision making proposed by economists and psychologists, and that bottom-up information from inverting this theory, given observations of behavior, is integrated with top-down prior knowledge of the space of goals to allow ra- tional Bayesian inference of goals from behavior. The inverse planning framework includes many specific models that differ in the complexity they assign to the be- liefs and desires of agents. Prior knowledge of the space of other agents’ goals is necessary for induction, and in this pa- per, we will present and test several models that differ in their representations of goal structure. Our experimental paradigm tests each model with a wide range of action trajectories in a simple space for which our models make fine-grained predic- tions. (Our stimuli resemble those of Gergely et al. (1995)). Some of these stimuli display direct paths to salient goals, and have simple intentional interpretations. Other stimuli display more complex behaviors, which may not have simple inten- tional interpretations. These sorts of trajectories allow us to distinguish between alternative models that differ in their rep- resentation of complex goal structure. By varying the length of the trajectories, we measure how subjects’ goal inferences change over time, and by eliciting both online and retrospec- tive inferences, we measure how subjects integrate informa- tion over time. To illustrate the space of models we present, consider the introductory example. Each of the three queries raised about Infants and adults are adept at inferring agents’ goals from in- complete or ambiguous sequences of behavior. We propose a framework for goal inference based on inverse planning, in which observers invert a probabilistic generative model of goal-dependent plans to infer agents’ goals. The inverse plan- ning framework encompasses many specific models and rep- resentations; we present several specific models and test them in two behavioral experiments on online and retrospective goal inference. Keywords: theory of mind; action understanding; Bayesian inference; Markov Decision Processes Introduction A woman is walking down the street, when suddenly she pauses, turns, and begins running in the opposite direction. Why? Is she crazy? Did she complete an errand unknown to us (perhaps dropping off a letter in a mailbox) and rush off to her next goal? Or did she change her mind about where she was going? These inferences derive from attributing goals to the woman and using them to explain her behavior. Adults are experts at inferring agents’ goals from obser- vations of behavior. Often these observations are ambiguous or incomplete, yet we confidently make goal inferences from such data many times each day. Developmental psychologists have shown that infants also perform simple forms of goal inference. In experiments using live-action stimuli, Wood- ward found evidence that 6-month old infants attribute goals to human actors, and look longer when subsequent behav- ior is inconsistent with the old goal (1998). Meltzoff (1995) showed that 18-month olds imitate intended acts of human actors rather than accidental ones, and Csibra and colleagues found evidence that infants infer goals from incomplete tra- jectories of moving objects in simple two-dimensional anima- tions (Csibra, Bir´o, Ko´os, & Gergely, 2003), both suggesting that children infer goals even from incomplete actions. The apparent ease of goal inference masks a sophisticated probabilistic induction. There are typically many goals logi- cally consistent with an agent’s actions in a particular context, and the apparent complexity of others’ actions invokes a con- fusing array of explanations, yet observers’ inductive leaps to likely goals occur effortlessly and accurately. How is this feat of induction possible? A possible solution, proposed by several philosophers and psychologists, is that these inferences are enabled by an intu- itive theory of agency that embodies the principle of rational- ity: the assumption that rational agents tend to achieve their desires as optimally as possible, given their beliefs (Dennett, 1987; Gergely, N´adasdy, Csibra, & Bir´o, 1995). However,