Foraging for Alternatives: Ecological Rationality in Keeping Options Viable

Foraging for Alternatives: Ecological Rationality in Keeping Options Viable Hansjorg Neth, Neele Engelmann & Ralf Mayrhofer {hneth|neele.engelmann|rmayrho}@uni-goettingen.de Cognitive and Decision Sciences Georg-Elias-Muller Institute of Psychology University of Gottingen, Germany Abstract Do we invest irrational amounts of effort into keeping options viable, or do we manage available and threatened options in an adaptive fashion? To ask and answer this question, we advo- cate an approach that considers the dynamic properties of deci- sion environments. By linking the exploration-vs.-exploitation dilemma to animal foraging, we show that preserving and abandoning options can both be adaptive. Specifically, peo- ple should stay and abandon options in progressive environ- ments, and leave and seek alternatives in exhaustive environ- ments. We extend a multi-arm bandit problem with threatened options by a manipulation of environmental expectations. Our findings show that people are highly sensitive to environmental assumptions and small payoff differentials. This replicates the original effect, but may explain the apparently irrational ten- dency to keep options open as an ecologically rational adapta- tion. Keywords: exploration vs. exploitation; multi-arm bandit; multi-tasking; switch costs; animal foraging. Should I stay or should I go now? If I go there will be trouble, and if I stay it will be double. The Clash Introduction Having a choice between multiple options confronts us with a dilemma: We desire the availability of options, but keep- ing options open often comes with costs. For instance, the simultaneous pursuit of several careers, research goals, or ro- mantic relationships can reduce the success or satisfaction en- joyed by any particular one. The dilemma between freedom of choice and the costs of preserving options is aggravated by the fact that options tend to deteriorate or disappear when we fail to invest enough effort or resources (e.g., attention, time, or money) into their availability. In an intriguing study, Shin and Ariely (2004) asked whether options are valued differently when they are in dan- ger of disappearing. The availability of multiple options was operationalized by playing a virtual “door game” in which participants first chose one of three doors to enter a room. From then on, participants could either earn a monetary re- ward by pressing a button within the chosen room or select a different door to switch to another room. As the overall bud- get of actions was limited to 100 clicks, switching between rooms incurred the opportunity cost of forgoing one payoff in the current room. In one condition, the three alternative rooms were continuously available. A second condition im- posed an additional threat of disappearing options by gradu- ally shrinking unused doors. On any click within a room or on the door to another room the two unchosen doors would lose 1/15th of their original width. This implies that the access to a room would effectively vanish if it has not been chosen within the 15 most recent clicks. Shin and Ariely’s (2004) main result was that the threat of losing options caused participants to invest additional efforts into their availability. Crucially, the costs of keeping doors open exceeded the potential benefits conveyed by them. Thus, people exhibited a strong and seemingly irrational tendency to keep poor options viable — a phenomenon that the authors attributed to an aversion to the loss of options. An analysis of the original study reveals an extremely de- manding task even when options are constantly available. Sequential decision environments involving payoff functions that must be estimated on the basis of limited experience tran- scend the realms of risky choice and require decision mak- ing under uncertainty (Sims, Neth, Jacobs, & Gray, 2013). Formally, the simultaneous availability of three rooms consti- tutes a multi-armed bandit problem (Banks, Olson, & Porter, 1997) in which participants must trade-off the exploration of unfamiliar choices with the exploitation of familiar ones to maximize the gain from probabilistic payoff functions (see Lee, Zhang, Munro, & Steyvers, 2011; Steyvers, Lee, & Wa- genmakers, 2009; Sutton & Barto, 1998, for formal models). In the present paradigm, this dilemma is aggravated by im- plicit but substantial switch costs that are paid by forgoing one payoff whenever entering another room. A further dif- ficulty of the original study was that the payoffs of all three rooms were sampled randomly from extremely similar payoff distributions. In fact, for the first 3 of 4 experiments the op- tions’ payoffs differed merely in their distributional form and type, but were equal in expected value. By contrast, the avail- ability of three separate doors suggested that people actually had a choice. If participants erroneously expected that the op- tions differed in some relevant aspect, the high similarity and overlap of experienced payoffs rendered the exploration task even more demanding. Thus, the threat of losing options was added to a non-trivial array of challenging task components. In a follow-up study that manipulated the difference between the payoff distributions of nine options, Ejova, Navarro, and Perfors (2009) showed that people are more willing to walk away from inferior options when the discrim- inability between options increases. This suggests that the dysfunctional aversion to option loss occurs mostly in situa- tions that are difficult to explore, whereas people are willing and able to sacrifice options when superior ones are available and identifiable.