Adjusting the Spanner: Testing an Evidence Accumulation Model of Decision Making

Adjusting the Spanner: Testing an Evidence Accumulation Model of Decision Making Ben R. Newell (ben.newell@unsw.edu.au) and Patrick Collins (patrickc@unsw.edu.au) School of Psychology, University of New South Wales Sydney, NSW, 2052, Australia Michael D. Lee (mdlee@uci.edu) Department of Cognitive Sciences, University of California, Irvine Irvine, CA, 92697-5100 Abstract An experiment examined two aspects of performance in a multi-attribute inference task: i) the effect of stimulus pre- sentation format (image or text) on the adoption of deci- sion strategies; and ii) the ability of an evidence accumula- tion model, which unifies take-the-best (TTB) and rational (RAT) strategies, to explain participants’ judgments. Presen- tation format had no significant effect on strategy adoption at a group level. Individual level analysis revealed large intra- participant consistency, including some participants who con- sistently changed the amount of evidence considered for a de- cision as a function of format, but wide inter-participant differ- ences. A unified model captured these individual differences and was preferred to the TTB or RAT models on the basis of the minimum description length model selection criterion. Keywords: Decision-making; Multi-cue inference; Heuris- tics; Take the best; Sequential sampling Log−Odds Evidence A B The idea that decision makers have at their disposal a va- riety of ‘tools’ or strategies that can be selected for particu- lar tasks has proved popular in the literature (Gigerenzer & Todd, 1999; Payne, Bettman, & Johnson, 1990; Rieskamp & Otto, 2006). The ‘adaptive toolbox’ metaphor proposed by Gigerenzer and colleagues is a good example of such an ap- proach. Proponents suggest that decision makers have access to a “collection of specialized cognitive mechanisms that evo- lution has built into the mind for specific domains of inference and reasoning” (Gigerenzer & Todd, 1999, p. 30). One of the key mechanisms or heuristics in this toolbox is the ‘take-the- best’ algorithm (TTB), a heuristic for choosing between two alternatives. The defining feature of TTB is that it terminates information search once a single cue that discriminates be- tween alternatives has been discovered. In this sense, TTB differs markedly from ‘rational’ decision models that advo- cate complete information search and optimal weighting of information. Despite the impressive success of TTB in simulation stud- ies, such as its ability to perform well against computation- ally intensive models (Gigerenzer & Goldstein, 1996), em- pirical studies seeking evidence that participants adopt TTB are more equivocal (Br¨oder, 2000; Newell & Shanks, 2003; Newell, Weston, & Shanks, 2003). In many experiments the results suggest that some people make choices consistent with TTB some of the time but a significant proportion of partici- pants adopt strategies that violate all or some of TTB’s rules, especially the ‘single discriminating cue’ stopping rule. Start Cue 1 Cue 2 Cue 3 Cue 4 Cue 5 Cue 6 Cue 7 Cue 8 Cue 9 Cue Sampling Figure 1: The unified sequential sampling model, showing the accumulated evidence as nine cues are sampled in validity order, and TTB-consistent (top) and RAT-consistent (bottom) decision thresholds. In an attempt to account for this wide individual variabil- ity Newell (2005) suggested an alternative metaphor—an ad- justable spanner (or wrench)—in which the width of the jaws represents the amount of evidence a person accumulates be- fore making a decision. The important feature of an evidence accumulation model for two-alternative choice problems is that it can mimic the performance of TTB’s stopping rule, or a strategy that incorporates more evidence, by adjusting the evidence required before a decision is made. Thus, one way of explaining individual variability is to suggest that all par- ticipants use an evidence-accumulation model but that some require greater amounts of evidence than others before mak- ing their decisions. Another appealing aspect of this model is that a ‘single tool’ circumvents the thorny issue of tool selec- tion (cf. Rieskamp & Otto, 2006). Lee and Cummins (2004) presented a formal instantiation of such an evidence accumulation model, which they pro- posed as a unification of TTB and rational models. As shown