More Likely but Less Probable: the Construction of Relative versus Absolute Belief the Construction of Absolute and Relative Belief

In this paper we document two systematic anomalies in relative and absolute likelihood judgment. First, we provide examples in which event A is judged “more likely” than event B but event B is assigned a higher probability than event A, violating procedure invariance. Second, we provide examples in which event A is judged more likely to occur than event B while event not-A is also judged more likely than event not-B, violating ordinal complementarity. We attribute these “belief reversals” to the complement neglect hypothesis: evidence for complementary events is underweighted in relative but not absolute likelihood judgment. We advance the “Contingent Weighting of Support” (CWS) model in which support theory is embedded within the contingent weighting model, and derive conditions under which each kind of belief reversal is expected to occur. We then fit this model to data in two experiments, and provide direct evidence of complement neglect in relative but not absolute likelihood judgment. Ordering Beliefs page 3 The Construction of Absolute and Relative Belief 1. Introduction. People are frequently called on to evaluate the relative likelihood of events. For instance, an investor might assess which of several mutual funds has the best chance of outperforming the market; a doctor might be asked which of three treatment options is most likely to cure a disease; a prospective buyer might wonder which of two car models is less likely to break down. Whether we infer a person’s belief from his or her choices, explicit probability estimates, or verbal statements that qualify belief (e.g., “It is fairly likely that.”), we rely on several implicit assumptions to derive consensual meaning from these data. Among the most basic assumptions is procedure invariance: the belief ordering of two events, A and B, should coincide under normatively equivalent elicitation modes. Thus, absolute and relative expressions of belief should be consistent with one another. For instance, when a weather forecaster says she thinks there is a 70% chance of rain tomorrow in Seattle and a 40% chance of rain tomorrow in Los Angeles, we assume that she would also say that she thinks it is “more likely” to rain tomorrow in Seattle than Los Angeles. Second, the ordering of beliefs of any pair of events should be the reverse of the ordering of their complements, an assumption that we label ordinal complementarity. Thus, we expect the forecaster to say she thinks it is more likely to rain in Atlanta than Boston if and only if she would also say that it is more likely to remain dry in Boston than Atlanta. The assumptions of procedure invariance and ordinal complementarity seem to be unassailable, if not tautological. In fact, axiomatic treatments of subjective probability do not typically distinguish between absolute and relative likelihood judgment, and they take ordinal Ordering Beliefs page 4 complementarity for granted (Fishburn, 1986; Krantz, Luce, Suppes & Tversky, 1971). 2 However, as we will demonstrate, both assumptions can fail in systematic and predictable ways. These violations provide important clues about how people weight evidence when making absolute and relative likelihood judgments. In this paper we document several examples of violations of procedure invariance and ordinal complementarity, develop a model of belief construction that accommodates them, fit this model and measure its parameters. To do so we embed support theory (Tversky& Koehler, 1994; Rottenstreich & Tversky, 1997) within the contingent weighting model (Tversky, Sattath & Slovic & 1988) to develop a new model that characterizes the construction of absolute and relative likelihood judgment. The psychology of likelihood judgment. Research on heuristics and biases (Tversky & Kahneman, 1974; Kahneman, Slovic & Tversky, 1982; Gilovich, Griffin & Kahneman, 2002; Shah & Oppenheimer, 2008) asserts that people judge the relative likelihood of events by evaluating a substitute attribute such as the relative ease with which instances of each event come to mind or the relative similarity of each event to a relevant prototype (Frederick & Kahneman, 2002; Kahneman, 2003). For instance, a person may judge the likelihood of rain to be greater in Seattle than in Los Angeles because recent instances of rain are easier to recall in Seattle than Los Angeles. Although studies of judgmental heuristics have been useful in predicting and explaining relative likelihood judgment, they do not explain how people quantify or qualify their absolute For instance, in Fishburn’s (1986) review of the axioms of subjective probability he sets up a system of axioms defined using the > operator so that A > B is read as “event A is (regarded by the individual as) more probable than event B,” but he does not explicitly distinguish absolute from relative probability judgment. Fishburn does explicitly advance ordinal complementarity as an axiom that he says “might quality [as] . so obvious and uncontroversial as to occasion no serious criticism” (p. 336). Likewise, Krantz, Luce, Suppes & Tversky (1971) define the “≥” operator as “qualitatively at least as probable as” and defer “debates about the meaning of probability” as being “in reality, about acceptable empirical methods to determine ≥” (p. 200). They later articulate ordinal complementarity as Lemma 4 that follows from other axioms that are necessary for a representation of qualitative probability (see pp. 203, 212). Ordering Beliefs page 5 degree of belief, which is required for many choices. For instance, the decision whether or not to carry an umbrella requires one to assess how likely it is to rain today; the decision whether or not to purchase an insurance policy requires one to estimate how likely one is to make a claim; the decision whether or not to accept a legal settlement offer requires one to estimate how likely one is to prevail in court. To model absolute likelihood judgment, support theory (Tversky & Koehler, 1994; Rottenstreich & Tversky, 1997) distinguishes between events and descriptions of events, called “hypotheses. ” Support theory conceives of probability as a judgment of the proportion of evidence favoring a focal hypothesis (e.g., “rain in Seattle next Tuesday”) relative to its complement (“no rain in Seattle next Tuesday”). People may recruit evidence for focal and complementary hypotheses using judgmental heuristics, explicit arguments, or objective data. For instance, the probability of rain in Seattle might be estimated by comparing the ease with which one can imagine a day with rain to the ease with which one can imagine a day without rain. If the former is equally easy as the latter then the probability might be estimated as 1⁄2 (i.e., odds of 1:1). In support theory evidence for the focal hypothesis and evidence for the complementary hypothesis receive equal weight in probability judgment. In contrast, research on heuristics such as availability and representativeness suggests that judgment of relative likelihood (e.g., “where is it more likely to rain? ”) can be assessed by comparing evidence for each focal hypothesis (e.g., “rain” in Seattle versus “rain” in Los Angeles) without reference to corresponding complementary hypotheses (days without rain in Seattle/Los Angeles), though this research does not mention explicitly the neglect of complementary hypotheses. 3 In our application of support theory we will not distinguish between different descriptions of the same event, so this distinction will not be especially relevant for most of this paper. However, we will return to the distinction between events and their descriptions in the discussion. Ordering Beliefs page 6 The notion that complementary hypotheses receive less weight in relative than absolute likelihood judgment is consistent with prior literature in judgment and decision making. Research on the compatibility principle holds that the weight that a particular stimulus feature receives is enhanced by its compatibility with the response mode (Tversky, Sattath & Slovic, 1988; see also Goldstein & Einhorn, 1987; Mellers, Ordóñez, & Birnbaum, 1992). Assessment of absolute likelihood, particularly when made on a percentage or fractional probability scale, is compatible with proportion judgments. Thus, it may be natural to consider the proportion of total evidence favoring the focal event when making such assessments, as in support theory. Assessment of relative likelihood, however, is more compatible with a comparison of evidence strength for the events being contrasted (e.g., whether it is easier to recall days with rain recently in Los Angeles or Seattle) than a comparison proportions of evidence favoring each target event over its corresponding complement. Complement neglect is also consistent with the observation that more prominent attributes tend to be afforded greater weight for tasks whose goal is to differentiate among options (Fischer & Hawkins, 1993; Fischer, Carmon, Ariely & Zauberman, 1999). In this case the focal hypotheses (e.g., rain in Seattle and Los Angeles) are more salient than the corresponding complementary hypotheses (i.e., no rain in Seattle and Los Angeles) because the rainy scenarios are explicitly mentioned. The notion that people afford greater weight or attention to focal information relative to contrary information, even when the alternative information is equally diagnostic, has precedence in literatures ranging from comparative social judgments (Chambers & Windchitl, 2004) to judgments of subjective well-being (Wilson, Wheatley, Meyers, Gilbert, & Axsom 2000) to consumer choice (Hoch, 1985) to judgmental confidence (Koehler, 1991) to the judged probability of grouped hypotheses (Brenner & Rottenstreich, 1999). We refer to the Ordering Beliefs page 7 tendency to underweight complementary events in relative likelihood judgment as the complement neglect hypothesis. When a judge has roughly equivalent knowledge concerning events unde