On Straight TRACS: A baseline bias from mental models

On Straight TRACS: A baseline bias from mental models Kevin Burns (kburns@mitre.org) The MITRE Corporation, 202 Burlington Road Bedford, MA 01730-1420 USA Abstract TRACS (Tool for Research on Adaptive Cognitive Strategies) is a family of games played with a special deck of two-sided cards (see www.tracsgame.com). TRACS has the advantage of being both mathematically tractable to theoretical analysis and psychologically relevant to practical applications. The simplest game, called Straight TRACS, is a series of choices where the player must turn over one of two cards to match a third card. The object is to make the most matches on a trip through the deck. The challenge is to track the changing odds in order to make the best choices. We performed experiments and simulations to measure human performance in this probabilistic and dynamic task. We present our finding of a Baseline Bias, in which subjective odds are (incorrectly) anchored to the baseline odds. This is an interesting result because it is contrary to other well-known biases, such as Gambler’s Fallacy, in which subjective odds are (incorrectly) not anchored to the baseline odds. We propose a theory of mental models to reconcile our finding with previous research on heuristics and biases. Introduction A dilemma of decision research is to obtain the rigors of controlled experimentation yet maintain some relevance to practical applications. Our approach is a synthetic task environment (Gray, in press) called “TRACS” (Burns, 2001a) that replicates the cognitive challenges of naturalistic decision-making (Klein, 1998), including probabilistic risk assessment and dynamic resource allocation. TRACS is both a unique game and a useful tool (Burns, 2001b). From a mathematical perspective, TRACS is unique because it is played with a special deck of two-sided cards, and because it has extensible rules that allow the same game to be played alone or with others. Unlike standard playing cards, the backs of the cards provide clues to the fronts, and the deck contains unequal numbers of each card type (Table 1). Compared to Poker and other games of imperfect information, the two-sided cards make TRACS more tractable to theoretical analysis of optimal solutions. Compared to Chess and other games of perfect information, the two-sided cards make TRACS more relevant to diagnoses and decisions in practical domains like business, medicine and warfare. From a psychological perspective, TRACS is useful because it provides a naturalistic micro world for experiments and simulations. Unlike other approaches to research on probabilistic reasoning, which often employ verbal stimuli in the form of static questions, TRACS employs graphical stimuli in a game of dynamic situations. This reduces artificial framing effects (see Nickerson, 1996) and introduces realistic temporal context. We are using TRACS to perform experiments on human subjects and to perform simulations with software agents. Our experiments are designed to elicit cognitive strategies and our simulations are designed to evaluate these strategies against normative standards. Taken together, our experiments and simulations allow us to build and test models of cognitive competence that are relevant to practical applications in command and control (Burns, 2001c). This paper reports on our first experiment and simulations using the simplest version of the game, called Straight TRACS. We explain the game, discuss our experiment and present our finding of a Baseline Bias. We also propose a theory of mental models to reconcile our finding with previous research on heuristics and biases. The Game Straight TRACS is a solitaire game played with 24 two- sided cards (Table 1). The backs of the cards, called “tracks”, show black shapes (triangle, circle or square). The fronts of the cards, called “treads”, show colored sets (Red or Blue) of these same shapes. Table 1 shows the distribution of shape/color (track/tread) cards in the deck. This distribution defines the baseline odds. For example, at the start of the game, a triangle track is likely (6/8 = 75%) to be Red, a square track is likely (6/8 = 75%) to be Blue and a circle track is 50-50. However, during the game, the odds change as the deck is depleted. Table 1: Distribution of cards in the deck. The backs are called “tracks” and the fronts are called “treads”. # of Cards Front (tread) Back (track) Red Red Red Blue Blue Blue