Measuring the Crowd Within: Probabilistic Representations Within Individuals

Psychological Science, Short Report, 2008. 19, 645-647. (in press version): This manuscript may differ from the final published version Measuring the crowd within: probabilistic representations within individuals. EDWARD VUL Massachusetts Institute of Technology HAROLD PASHLER University of California, San Diego A crowd often possesses better information than do the individuals it comprises. For example, if people are asked to guess the weight of a prize- winning ox (Galton, 1907), the error of the average response is substantially smaller than the average error of individual estimates. This fact, which Galton interpreted as support for democratic governance, is responsible for the success of polling the audience in the television program “Who Wants to be a Millionaire” (Surowiecki, 2004) and for the superiority of combined over individual financial forecasts (Clemen, 1989). Researchers agree that this wisdom-of-crowds effect depends on a statistical fact: The crowd's average will be more accurate as long as some of the error of one individual is statistically independent of the error of other individuals—as seems almost guaranteed to be the case. Whether a similar improvement can be obtained by averaging two estimates from a single individual is not, a priori, obvious. If one estimate represents the best information available to the person, as common intuition suggests, then a second guess will simply add noise, and averaging the two will only decrease accuracy. Researchers have previously assumed this view and focused on improving the best estimate (Hirt & Markman, 1995; Mussweiler, Strack, & Pfeiffer, 2000; Stewart, 2001). Alternatively, single estimates may represent samples drawn from an internal probability Address correspondence to Edward Vul, Department of Brain and Cognitive Science, Massachusetts Institute of Technology, 77 Massachusetts Ave. 46- 4141, Cambridge, MA 02139, e-mail: evul@mit.edu. distribution, rather than deterministic best guesses. According to this account, if the internal probability distribution is unbiased, the average of two estimates from one person will be more accurate than a single estimate. Ariely et al. (2000) predicted that such a benefit would accrue from averaging probability judgments within one individual, but did not find evidence of such an effect. However, probability judgments are known to be biased toward extreme values (0 or 1), and averaging should not reduce the bias of estimates; if guesses are sampled from an unbiased distribution, however, averaging should reduce error (variance; Laplace, 1812/1878; Wallsten, Budescu, Erev, & Diederich, 1997). Probabilistic representations have been postulated in recent models of memory (Steyvers, Griffiths, & Dennis, 2006), perception (Kersten & Yuille, 2003), and neural coding (Ma, Beck, Latham, & Pouget, 2006). It is consistent with such models that responses of many people are distributed probabilistically, as shown by the wisdom-of-crowds effect. However, despite the theoretical appeal of these models, there has been scant evidence that, within a given person, knowledge is represented as a probability distribution. Finding any benefit of averaging two responses from one person would yield support for this hypothesis. METHOD We recruited 428 participants from an Internet- based subject pool and asked them eight questions probing their real-world knowledge (derived from The World Factbook, Central Intelligence Agency, 2007; e.g., “What percentage of the world's airports are in the