Explicit Bayesian Reasoning with Frequencies, Probabilities, and Surprisals

Explicit Bayesian Reasoning with Frequencies, Probabilities, and Surprisals Heather Prime (heath.prime@gmail.com) Department of Human Development and Applied Psychology, Ontario Institute for Studies in Education/University of Toronto, 252 Bloor Street West Toronto, ON M5S 1V6 Canada Thomas R. Shultz (thomas.shultz@mcgill.ca) Department of Psychology and School of Computer Science, McGill University, 1205 Penfield Avenue Montreal, QC H3A 1B1 Canada the use of frequencies instead of probabilities to describe reasoning problems and the use of surprisals, the degree of being surprised, instead of probabilities to formulate responses. Abstract To explore human deviations from Bayes’ rule in numerically explicit problems, prior and likelihood probabilities or frequencies are manipulated and their effects on posterior probabilities or surprisals are measured. Results show that people use both priors and likelihoods in Bayesian directions, but the effect of likelihood information is stronger than that of prior information. Use of frequency information and surprisal measures increase deviations from Bayesian predictions. There is evidence that people do compute something like the standardizing marginal data term when asked for probability estimates, but not when asked for surprisal ratings. Bayesian Predictions Bayes’ rule specifies that the posterior probability of a hypothesis h given data d is equal to the prior probability of the hypothesis times the conditional likelihood of the data given the hypothesis divided by the marginal probability of the data, defined as the sum of prior by likelihood products over all hypotheses H: P ( h ) × P ( d | h ) P ( h | d ) = ∑ P ( h ′ ) × P ( d | h ′ ) Keywords: Reasoning under uncertainty; Bayes’ rule; rationality; base-rate neglect; probability; surprisal. Introduction h ′ ∈ H A pressing issue in contemporary cognitive science is whether people exhibit rationality in their inferences and learning under uncertainty, with rationality defined in terms of conformity to Bayes’ rule. Evidence for Bayesian conformity comes from a wide variety of domains including sensorimotor control (Kording & Wolpert, 2006), vision (Yuille & Kersten, 2006), conditioning (Courville, Daw, & Touretzky, 2006), induction and inference (Tenenbaum, Griffiths, & Kemp, 2006), and language (Chater & Manning, 2006). In each of these areas, Bayesian models account for a wide range of inferential phenonmena. An unresolved problem is that these contemporary conclusions of Bayesian rationality appear to conflict with earlier Nobel-Prize-winning work showing that people are rather poor Bayesians, subject to such biases as base-rate neglect (Kahneman, Slovic, & Tversky, 1982; Kahneman & Tversky, 1996; Tversky & Kahneman, 1974, 1981). The purpose of this paper is to contribute to the resolution of this discrepancy between demonstrated Bayesian successes and failures. It may be tempting to explain this discrepancy by noting that the successes mainly involve performances in which the Bayesian ideal is only implicit, whereas the failures mainly involve more explicit reasoning with numerical problems. But why this implicit-explicit distinction would matter for Bayesian success would need to be explained for this argument to be successful. The present work focuses on factors that might be expected to improve Bayesian performance on problems that are typically described as explicit. These factors include One possibility is that people reason, not with probabilities and Bayes’ rule, but rather with surprisals. A surprisal is mathematically defined in information theory as the negative of the log to the base 2 of the probability of the event (Cover & Thomas, 1991): S ( p ) = − log 2 p More generally, the log in Equation 2 can be computed to any base and multiplied by a constant. Here, we simplify by considering the constant to be 1 and using the base 2, which is suitable for the binary decisions that we consider. The idea that people might reason with surprisals is based on the intuition that people do not often know probabilities and how to compute with them, but they do know whether and how much they are surprised by events. For example, estimate the probability that: 1. Middle-eastern terrorists will destroy the NY trade towers, the White House, the US Capitol, and the Pentagon in the same hour. (Best before 9-11). 2. A gunman will enter a one-room Amish school house and kill 7 little girls. 3. A modern highway overpass will collapse early one weekend morning killing the occupants of a car passing underneath. Although such probabilities are difficult to compute or estimate, people do know that they are very surprised when such events happen. Perhaps surprisals would capture people’s intuitions about events more naturally than probabilities would.