Learning to adapt evidence thresholds in decision making

Learning to Adapt Evidence Thresholds in Decision Making Ben R. Newell (ben.newell@ unsw.edu.au) School of Psychology, University of New South Wales, Sydney, Australia Michael D. Lee (mdlee@uci.edu) Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, 92697-5100, USA to rather pessimistic conclusions. Broder and Schiffer (2006) considered environments in which either a compensatory strategy (one which weights and integrates all cue information) or a noncompensatory strategy (one which considers only a subset of the information) was optimal with respect to the expected monetary pay-off. Participants detected the appropriate strategy in whichever environment they encountered initially, but when the environment changed (mid-way through the experiment) most participants retained the same strategy despite its sub-optimality in the new environment. Broder and Schiffer explained these results in terms of the application of maladaptive routines: participants used a ‘top-down’ deliberative mode of thinking in the initial phase to work out the appropriate strategy, but then when this appeared to work successfully slipped into a more ‘bottom-up’ routine mode and thus failed to test the consequences of applying the strategy on each trial when the environment changed. In a similar vein, Rieskamp (2006) used decision environments in which the lexicographic strategy Take- the-Best (TTB; Gigerenzer & Goldstein, 1996) was more or less adaptive (in terms of expected monetary pay-off) than a weighted additive strategy (WADD; e.g., Payne, Bettman & Johnson, 1993). In the noncompensatory environment TTB led to 83% correct predictions compared to WADD’s 60%; and these values were reversed in the compensatory environment. Participants who transitioned from the non-compensatory environment to the compensatory one showed a distinct inertia, resisting the change to the more adaptive strategy. Those transitioning in the opposite direction showed more of a change (the fit of TTB increased relative to WADD in the final block of trials) but this was tempered by a less pronounced adoption of WADD in the first half of the experiment. The inertia effect was predicted and explained by the Strategy Selection Learning model of Rieskamp and Otto (2006). The model states that adaptation will be very slow because a strategy that is successful in the initial environment will accrue considerable reinforcement, and this reinforcement will only gradually diminish in the novel environment on the rare occasions when a participant explores the potential of the competing strategy. Abstract A basic challenge in decision-making is to know how long to search for information, and how to adapt search processes as performance, goals, and the nature of the task environment vary. We consider human performance on two experiments involving a sequence of simple multiple-cue decision-making trials, which allow search to be measured, and provide feedback on decision accuracy. In both experiments, the nature of the trials changes, unannounced, several times. Initially minimal search is required, then more extensive search is required, and finally only minimal search is again required to achieve decision accuracy. We find that people, considered both on aggregate, and as individuals, are sensitive to all of these changes. We discuss the theoretical implications of these findings for modeling search and decision-making, and emphasize that they show adaptation to an external error signal must be accompanied by some sort of internal self-regulation in any satisfactory account of people’s behavior. Keywords: evidence accumulation, heuristics, decision- making, learning, self-regulation Introduction A problem faced commonly by decision makers is determining how much information to incorporate into a decision. Some decisions are trivial (e.g., choosing a breakfast cereal), but some more important (e.g., choosing a mate), and consequently the amount of information or evidence examined prior to deciding will vary. One way to model this variance is to suggest that people sample evidence sequentially and adjust the amount of evidence they consider according to a decision threshold. Inherent in this conception is that thresholds will vary not just between decisions but also between individuals (Lee & Cummins, 2004; Vickers, 1979). Newell (2005) suggested an ‘adjustable spanner’ (or wrench) to capture this idea; a spanner in which the width of the jaws represents the amount of evidence a person accumulates before making a decision. In this paper we develop this perspective by examining how people learn to adjust their evidence thresholds in dynamic decision environments. Adapting to changing environments Some recent studies examining adaptation to changes in the statistical structure of decision environments have led