A preference for the unpredictable over the informative during self-directed learning

A preference for the unpredictable over the informative during self-directed learning Doug Markant (markant@mpib-berlin.mpg.de) Center for Adaptive Rationality Max Planck Institute for Human Development Lentzeallee 94, 14195 Berlin, Germany Todd M. Gureckis (todd.gureckis@nyu.edu) New York University Department of Psychology 6 Washington Place, New York, NY 10003 USA Abstract mimic IG under common kinds of problems where positive evidence is rare but highly informative (Klayman & Ha, 1987; Navarro & Perfors, 2011). The potential information gained from asking a question and one’s uncertainty about the answer to that question are not always the same. For example, given a coin that one believes to be fair, the uncertainty a person has about the outcome of flipping that coin is high, but either outcome is unlikely to make them believe that the coin is biased (i.e., the “information gain” of that observation is low). In the present paper we show that people use a simple form of predictive uncertainty to guide their information sampling decisions, a strategy which is often equivalent to maximizing information gain, but is less efficient in environments where potential queries vary in their reliability. We conclude that a potentially powerful driver of human information gathering may be the inability to predict what will happen as a result of an action or query. Similarly, under some conditions IG is equivalent to a sim- ple preference for observations whose outcomes are difficult to predict, which is captured by a sampling model we refer to as label entropy (LE, where a “label” is the feedback re- ceived from an observation). Specifically, when hypotheses are deterministic (i.e., they predict the outcome of a given observation with probability 0 or 1), IG and LE make iden- tical predictions as to the value of different queries (Markant et al., in revision). The distinction between IG and LE can be understood intuitively in terms of a possibly biased coin toss. If one is confident that the coin is fair, then a single flip (heads or tails) is unlikely to change that belief. At the same time, it is difficult to predict what the outcome of the coin flip will be. Whereas evaluating the IG of a potential query implies a consideration of how a query might change one’s beliefs, LE only requires a person to evaluate how confidently they can predict its outcome in light of what they already know. Keywords: self-directed learning, active learning, infor- mation search How do people make decisions to gather information? One common approach for answering this question is to com- pare information gathering behavior of humans to norma- tive theories that objectively measure the information con- tained in particular queries. For example, information gain (IG) measures the usefulness of a potential query based on how much it is expected to reduce uncertainty. IG has been shown to account for sampling decisions in a number of search problems (Nelson, 2005), with recent work suggest- ing that it can also predict eye movements during visual search, including when searching for a target embedded in noise (Najemnik & Geisler, 2005), disambiguating the iden- tity of a shape (Renninger et al., 2007), and when learning the relevance of features for categorization (Nelson & Cot- trell, 2007). However, it is important to note that normative models (even when they fit well) do not necessarily reveal people’s underlying decision making process. For example, in many situations, simpler decision making strategies are consistent with IG, showing that judgments about the rationality of any single preference during information search can only be made with respect to the structure of the learning environ- ment (Klayman & Ha, 1987). For example, people often use positive testing (PT), a preference for positive instances or strongly predicted features. Although PT has often been por- trayed as a form of confirmation bias, it has been shown to In prior experiments using the task described in this pa- per, we found that people make sampling decisions that are consistent with IG (Gureckis & Markant, 2009; Markant & Gureckis, 2012), but in those tasks LE made identical predic- tions owing to the use of deterministic hypotheses. The goal of the current experiment was to differentiate between LE and IG by decoupling the amount of information conveyed by a query from the uncertainty about its outcome. One way to achieve this is for queries to vary in their reliability, that is, the probability of receiving accurate feedback about their out- come. Varying the reliability across potential queries allows us to differentiate possible models of human information sam- pling. Given an information source for which reliability is low, a learner should be highly uncertain about the out- come of querying it, while the expected information conveyed should be low. If people rely solely on their uncertainty about the outcome of a particular observation in order to judge its value, they should choose in accordance with LE. Alterna- tively, if they evaluate how informative a query is expected to be, taking into account any chance of incorrect feedback, they