Evaluating Theory of Mind in Question Answering

We propose a new dataset for evaluating question answering models with respect to their capacity to reason about beliefs. Our tasks are inspired by theory-of-mind experiments that examine whether children are able to reason about the beliefs of others, in particular when those beliefs differ from reality. We evaluate a number of recent neural models with memory augmentation. We find that all fail on our tasks, which require keeping track of inconsistent states of the world; moreover, the models’ accuracy decreases notably when random sentences are introduced to the tasks at test.1 1 Reasoning About Beliefs Possessing a capacity similar to human reasoning has been argued to be necessary for the success of artificial intelligence systems (e.g., Levesque et al., 2011). One well-studied domain that requires reasoning is question answering, where simply memorizing and looking up information is often not enough to correctly answer a question. For example, given the very simple scenario in Table 1, searching for the word “Mary” and returning a nearby word is not a correct strategy; instead, a model needs to recognize that Mary is currently at the second location (office and not the bathroom). Recent research has focused on developing neural models that succeed in such scenarios (Sukhbaatar et al., 2015; Henaff et al., 2017). As a benchmark to evaluate these models, Weston et al. (2016) released a dataset – Facebook bAbi – that provides a set of toy tasks, each examining a specific type of reasoning. For example, the scenario in Table 1 evaluates the capacity to reason using a single supporting fact. However, the bAbi tasks are already too simple for the current models. Only a few years after their release, existing 1 Code to generate dataset and replicate results is available at github.com/kayburns/tom-qa-dataset. models fail at only one or two (out of 20) tasks (Rae et al., 2016; Santoro et al., 2017). Moreover, all except two of the reasoning tasks in this dataset only require transitive inference (Lee et al., 2016). Mary went to the bathroom. John moved to the hallway. Mary travelled to the office. Where is Mary? A: office Table 1: A task from the bAbi dataset (Weston et al., 2016). People reason not just about their own observations and beliefs but also about others’ mental states (such as beliefs and intentions). The capacity to recognize that others can have mental states different than one’s own – theory of mind – marks an important milestone in the development of children and has been extensively studied by psychologists (for a review, see Flavell, 2004). Artificial intelligence (AI) systems will also require a similar reasoning capacity about mental states as they are expected to be able to interact with people (e.g., Chandrasekaran et al., 2017; Grant et al., 2017; Rabinowitz et al., 2018). However, the bAbi dataset does not include tasks that evaluate a model’s ability to reason about beliefs. Grant et al. (2017) created a bAbistyle dataset inspired by an influential experiment on the theory of mind called the Sally-Anne task (e.g. Baron-Cohen et al., 1985). Their goal was to examine whether the end-to-end memory network (Sukhbaatar et al., 2015) can answer questions such as “where does Sally think the milk is?” in situations that Sally’s belief about the location of milk does not match the reality. For example, Sally thinks that the milk is in the fridge but the milk is actually on the table. The dataset of Grant et al. (2017) provides a first step in designing benchmarks to evaluate the mental-state reasoning capacity of questionanswering models, but it is still limited in the types of reasoning it probes. For example, it ar X iv :1 80 8. 09 35 2v 1 [ cs .C L ] 2 8 A ug 2 01 8 only considered first-order beliefs (e.g., Sally’s belief about the location of milk). People also reason about second-order (and higher-order) beliefs (e.g., Anne’s belief about Sally’s belief about the location of the milk). More importantly, similarly to the bAbi dataset, success in each task is defined as correctly answering one question. This does not guarantee that a model has an understanding of the state of the world; in fact, even in developmental theory-of-mind experiments, children are asked a few questions (e.g., “where is milk really?”) to ensure that their correct answer reflects their understanding and is not simply due to chance. In this paper, we address these shortcomings by designing a new dataset that enables us to evaluate a model’s capacity to reason about different types of beliefs as well as whether it maintains a correct understanding of the world. To this end, we evaluate a number of different models that perform well on the bAbi tasks: the end-to-end memory network (Sukhbaatar et al., 2015), the multiple observer model (Grant et al., 2017), the recurrent entity network (Henaff et al., 2017), and RelationNetwork (Santoro et al., 2017). We find that none of these models succeed at our tasks, suggesting that they are not able to keep track of inconsistent states of the world, in particular when someone’s belief does not match the history or reality of a situation. 2 Theory of Mind Experiments Behavioral research shows that children gradually develop a theory of mind (for a review, see Gopnik and Astington, 1988). At the age of two, most children have an understanding of others’ desires and perceptions – if someone wants something, they will try to get it and if something is in their sight, they can see it. Children begin to understand others’ beliefs around the age of three, but this understanding is still limited. For example, they might not be able to reason that someone’s actions are a result of their beliefs. By the age of five, most children have a unified theory of mind and are able to represent and reason about others’ desires, perceptions, and beliefs. Developmental psychologists have designed various experimental paradigms to examine to what extent children are able to reason about others’ mental states. We use these experiments as guidelines for designing tasks to evaluate the reasoning capacity of question-answering models. We first explain these experiments. 2.1 The Sally-Anne Experiment The Sally-Anne false-belief experiment, proposed by Baron-Cohen et al. (1985), examines children’s ability to reason about others’ false beliefs, i.e., when someone’s belief does not match the reality. In this experiment, the participants observe two agents, Sally and Anne, with their containers, a basket and a box. After putting a marble in her basket, Sally leaves the room (and is not able to observe the events anymore). After Sally’s departure, Anne moves the marble to her box. Then, Sally returns to the room (see Figure 1). The participants are asked the following questions: • “Where will Sally look for her marble?”