Time-interval statistics adaptively modulate decision makers' willingness to wait for delayed outcomes

Time-interval statistics adaptively modulate decision makers’ willingness to wait for delayed outcomes Joseph T. McGuire (mcguirej@psych.upenn.edu) Department of Psychology, University of Pennsylvania 3720 Walnut Street, Philadelphia, PA 19104 USA Joseph W. Kable (kable@psych.upenn.edu) Department of Psychology, University of Pennsylvania 3720 Walnut Street, Philadelphia, PA 19104 USA Abstract The present work examines persistence in situations where de- lays are open-ended. From a normative standpoint, appropriate behavior in such situations depends on the statistical distribu- tion of possible delay lengths. Depending on this distribution it may be appropriate either to persist indefinitely or to give up after a short period of time. In a behavioral experiment, human participants experienced reward timing statistics that implied it was productive to adopt either a high or low level of persistence. Human decision makers were highly responsive to these statistical cues. In a condition where timing statis- tics implied patience was productive, participants performed exceptionally well, and had little difficulty in waiting for de- layed outcomes. In contrast, participants showed substantially lower willingness to wait when temporal statistics implied pa- tience was an inappropriate strategy. The results demonstrate that seemingly impatient behavior can arise as an adaptive re- sponse to the perceived statistics of the environment. Keywords: decision making; intertemporal choice; delay of gratification; dynamic inconsistency Introduction Persistence in pursuing delayed rewards is widely regarded as an important self-control skill (Ainslie, 1975; Mischel, Shoda, & Rodriguez, 1989). Likewise, the inability to tol- erate delay of gratification has been viewed as a source of maladaptive inconsistencies in choice. Theoretical analyses of intertemporal choice have focused primarily on situations where decision makers know in ad- vance how long a delay is scheduled to last (e.g., Ainslie, 1975). However, real-life decision makers routinely find themselves waiting through uncertain, open-ended delay in- tervals. When decision makers do not know how soon a re- ward will arrive, they face a nontrivial task of deciding how long to continue waiting. Open-ended delays occur when commuters wait for buses, when job-seekers wait for offers, and when sit-and-wait predators wait for prey. In each of these cases, a decision maker must continuously choose whether to keep waiting or move on to new opportunities. Several of the most fa- mous and compelling empirical examinations of delay-of- gratification behavior have involved delays that were open- ended from the decision maker’s point of view (Mischel & Ebbesen, 1970; Mischel, Ebbesen, & Zeiss, 1972; Mischel et al., 1989). Little is known about how decision makers cope with tim- ing uncertainty during intertemporal choice. From a norma- tive standpoint, as we show in detail below, the appropriate behavior depends on the statistical distribution of possible de- lay durations. The shape of this distribution determines how a decision maker’s expectation should change as time passes. For some distributions the anticipated delay grows steadily shorter over time, presumably increasing the reward’s present subjective value. For others, however, time-passage can actu- ally increase the expected remaining delay time. In this case a delayed outcome loses value over time. Giving up on the delayed outcome would be inappropriate in the former case, but is potentially justified in the latter. Here we present a behavioral experiment in which decision makers had the opportunity to wait for rewards in environ- ments that differed in terms of their timing statistics. Results suggest that individuals successfully learned about these tim- ing statistics through experience, and responded by making appropriate adjustments in their willingness to tolerate delay. Impulsivity and inability to tolerate delay often have a detrimental influence on human decision making. Neverthe- less, a potentially productive route to understanding these as- pects of behavior is to examine situations in which they are adaptive. It may turn out that mechanisms that support appro- priate responding in some situations are also responsible for maladaptive delay-of-gratification failure in others. As a gen- eral point, it is important to recognize that a decision maker’s computational-level objective is to calibrate behavioral per- sistence, not merely to maximize persistence in all cases. The present work relies on the assumption that an individ- ual’s temporal expectations take the form of a probability dis- tribution, not merely a point estimate. We therefore begin by reviewing evidence that supports this contention. Experience-based learning of time-interval distributions Considerable evidence suggests that decision-making organ- isms can encode and use information about full distributions of time intervals. A first category of evidence involves instrumental behav- ior under interval schedules of reinforcement. Fluctuations in response frequency are highly sensitive to the specific reward schedule in effect. If rewards are made available at fixed tem- poral intervals, response rates show a “fixed-interval scallop” (Gibbon, 1977), rising and falling as if to reflect subjective reward probability. Catania and Reynolds (1968) observed that pigeons’ response rates tracked reward hazard rates that