Do I know that you know what you know? Modeling testimony in causal inference

Do I know that you know what you know? Modeling testimony in causal inference Daphna Buchsbaum 1 , Sophie Bridgers 1 , Andrew Whalen Elizabeth Seiver, Thomas L. Griffiths, Alison Gopnik {daphnab, sbridgers, awhalen,seiver,tom griffiths, gopnik}@berkeley.edu Department of Psychology, University of California, Berkeley, Berkeley, CA 94720 USA Abstract but has not examined in detail how we make inferences about their credibility based on their causal statements. Here, we explore how people combine information from causal observations and testimony, both to make causal judgments and to evaluate the informants themselves. Recently, there has been a growing literature on how peo- ple, especially children, evaluate informants (e.g., Borckardt, Sprohge, & Nash, 2003; Corriveau, Meints, & Harris, 2009; Koenig & Harris, 2005), including how children integrate their prior knowledge with informant testimony (e.g., Jaswal, 2010; Jaswal & Markman, 2007). Integrating testimony with our observations is particularly challenging because multiple aspects of informants and their testimony contribute to the value of the information they provide, and to how much they should be trusted in the future. These aspects include the level of certainty informants express, their past accuracy, and their self-knowledge – how well their certainty reflects their true knowledge (for an exploration of a similar idea in the context of eye-witness testimony see Tenney, Small, Kondrad, Jaswal, & Spellman, 2011). The difficulty of com- bining information from what someone says and what we see is especially apparent when an informant’s statements appear to be incorrect. It could be that the informant is actually right, and our own observations were inaccurate. Alternatively, the informant may be a knowledgeable person who has simply misspoken, or she could truly be clueless. Finally, an infor- mant could express certainty or uncertainty about her knowl- edge, shedding a different light on her inaccuracy. Deciding whether an informant has erred, and if so, why, and whether to trust her in the future is therefore a complex problem. Bayesian modeling provides a mechanism for explicitly representing the contributions of different sources of informa- tion to judgments about causal structure and informant credi- bility. Previous work has used such models to explore the role of social observations in causal learning (e.g., Goodman et al., 2009), and to evaluate the role of informant knowledgeability and helpfulness (Shafto, Eaves, Navarro, & Perfors, in press). Here, we present a model that helps us evaluate the roles of observed cause and effect patterns, as well as an informant’s expressed certainty, current and past accuracy, and awareness of her own knowledge level, when making a causal inference. In this paper, we first review a study exploring how preschoolers combine information from informant testi- mony with conflicting information from observed causal data. Next, we introduce a computational model of causal inference from testimony that explicitly represents the roles of informant certainty, accuracy, and self-knowledge, as well as direct causal observations, allowing us to assess the We rely on both our own observations and on others’ testimony when making causal inferences. To integrate these sources of information we must consider an informant’s statements about the world, her expressed level of certainty, her previous accuracy, and perhaps her apparent self-knowledge – how accurately she conveys her own certainty. It can be difficult to tease apart the contributions of all these variables simply by observing people’s causal judgments. We present a computa- tional account of how these different cues contribute to a ratio- nal causal inference, and two experiments looking at adults’ inferences from causal demonstrations and informant testi- mony, focusing on cases where these sources conflict. We find that adults are able to combine social information with their own observations, and are sensitive to the reliability of each. Adults are also sensitive to the accuracy, certainty, and self- knowledge of the informant, a result confirmed by comparing predictions from models with and without these variables. Introduction People face challenging causal learning problems on a daily basis. They have a variety of information they can use to help solve these problems, including directly observed patterns of cause and effect, and social data such as others’ statements about existing causal relationships. Having multiple sources available should enhance our causal reasoning, but integra- tion can be difficult, especially when sources disagree. If an informant’s causal statements contradict our causal observa- tions, which source should we trust? Informants can be igno- rant, mistaken, even deceptive, so one might think we should always trust what we see over what we hear. Yet the world is unpredictable: Observing a phenomenon once does not mean we will reliably observe it again. How do we evaluate these sources and determine which to rely on? We attempt to better understand how people combine different sources of infor- mation when making causal inferences, how they integrate their own observations with conflicting testimony, and how this affects their future evaluation of the social informant. Informant testimony is a key type of social information that guides learning across domains, but the role of testi- mony in causal learning has not been extensively explored. Research on how we incorporate information from the social context into our causal judgments has shown that both children and adults are skilled causal learners, and can use information from social demonstration to inform their causal inferences (e.g., Kushnir, Wellman, & Gelman, 2008; Sobel & Sommerville, 2009; McGuigan, Makinson, & Whiten, 2011). This work has investigated how we learn by observing other people (e.g., Goodman, Baker, & Tenenbaum, 2009; Buchsbaum, Gopnik, Griffiths, & Shafto, 2011), and by ob- serving different types of people (e.g., Kushnir et al., 2008), 1 These authors contributed equally to this work.