The illusion of argument justification.

Disagreement is an inevitable part of our daily lives. We form opinions, take sides, and argue for our point of view. Whether it is a young child talking about which toy is best or a leading intellectual justifying a complex technical position, arguments are a key part of human interactions. To know if the arguments in support of opinions on important issues are sound, one important factor is the quality of argument that can be produced. Here, we propose an “illusion of argument justification” in which people overrate the quality of justification they can provide for their positions on controversial topics. We also provide evidence that emotional investment leads to a greater difference between initial appraisals of argument strength and actual persuasive force. We attempt to counteract the illusion through tacit interventions and then show that explicit consideration of alternative perspectives leads to debiasing. We are interested in whether accurate judgments of argument quality for personal positions are readily accessible. This metacognitive ability could provide insight into how arguments are understood and engaged. If people are poor predictors of the ability to justify their views, it would point to the concern that they are ill-equipped when they enter arguments. We do not aim to cover all possible factors on an argument, but for the purposes of these studies, we focus on the introspective accuracy of the ability to justify to an audience through argument. Other research on self-evaluation and metacognition suggests that this kind of self-assessment may pose a serious challenge. People overestimate the affective impact of future events (Wilson et al. 2000), perceive themselves as above average (Alicke et al. 1995; Dunning et al. 2003; Heady & Wearing, 1987), and inaccurately predict how long it will take to complete a task (Kahneman & Tversky, 1979; Buehler, Griffin, & Ross, 1994). More generally, people have a meta-bias, known as the “bias blind spot”, where they more readily see the effects of cognitive and motivational biases in others but not in themselves (Pronin, Lin & Ross, 2002). The blind spot is supported by valuing introspection (Pronin & Kugler, 2007) and by naive realism, the tendency to view one’s own subjective interpretation as the truth about reality (Griffin & Ross, 1991). Folk theories explaining phenomena are surprisingly incomplete (Wilson & Keil, 1998) and people are often unaware of the lack of depth in their understanding. In the context of reasoning about argumentation, we believe emotional investment could influence self-assessment about the ability to justify a position through argumentation. Self-assessment for argumentation might be especially difficult because emotional investment prevents objective self-examination. Arguments, especially those dealing with controversial issues, are qualitatively different from explanations. Arguments often come with deep ties to emotions and values. Thus, we may expect emotions and values to modulate self-assessments for arguments more than those for explanations even though both may have explanatory components. The perceived strength of an argument changes according to a variety of factors. One’s level of involvement leads to stronger attitudes. For example, opinions become more extreme after participants commit to express their opinions publicly (Jellison & Mills, 1969). Attitude commitment, as measured by certainty, likelihood of change, and extremity, is associated with increased intentions to act on that attitude, attitude polarization, resistance to opposing arguments, and biased elaboration (Pomerantz et al. 1995). Similarly, attitude embeddedness, which includes measures of importance of attitude, the attitude’s representativeness of values, and relevance to concept of self, is significantly associated with objective elaboration but also with biased memory, especially for arguments opposing one’s own attitude. These strong attitudes also tend to be stable: the greater the importance of an attitude, the less likely it is to change over time (Krosnick, 1988). Commitment to attitudes may also be strengthened through direct connection to values (Kristiansen & Zanna, 1988). Furthermore, consideration of normatively important values causes more processing, strengthened attitudes, and resistance to opposing arguments even on issues of little personal relevance (Blackenship &Wegener, 2008). Thus, strongly caring about an issue can polarize attitudes in several ways. Emotional investment also affects how people view arguments. In one study, participants first rated the strength of pro and con arguments for seven issues and then participated in a thought-listing task by responding to a randomized subset of the arguments in short sentences (Edwards & Smith, 1996). Based on previous ratings of feelings toward the issues, the participants were placed in either a low or high emotional conviction group. Participants with high emotional conviction on an issue rated opposing arguments significantly weaker, generated more overall arguments, and generated more redundant arguments to undermine the opposing view than did participants with low emotional conviction on an issue. These biases from emotional investment could inflate people’s assessment of their ability to actually articulate a coherent argument, especially if they are not fully aware of the redundancies in their own arguments and falsely see more detail and support where there is not. We therefore hypothesize that the more one cares about an issue, the stronger the illusion of argument justification. Ironically, while care and investment might be thought to lead to greater intellectual investment and understanding, the more prominent effect may merely be an illusion of having such a competence. Overview of Present Studies Self-testing the ability to justify through an argument requires sufficient time. One way to measure a possible illusion of argument justification is to observe how participants’ assessments of their own abilities change over time. To accomplish this, we adopted a time-sensitive measure that has been used in previous research on the illusion of explanatory depth (IOED), where people overestimate their ability to produce explanatory knowledge for mechanical objects and natural processes (Rozenblit & Keil, 2002). In the IOED research procedure, participants provided initial ratings for their level of understanding for a series of devices and phenomena without pausing excessively on any item. After writing out explanations for a subset of these items, participants rated their depth of understanding as significantly lower. This effect across time was observed for explanations and not for other types of knowledge, suggesting that the illusion is not due to general overconfidence. Moreover, the IOED is even stronger in young children and therefore seems to be a foundational cognitive bias (Mills & Keil, 2004). Others have used the methodology of IOED research to investigate common controversial topics like political disagreements (Alter, Oppenheimer, & Zemla, 2010; Fernbach et al, in press) but we applied it to people’s judgments of their ability justify through argumentation their position on these sorts of issues. We are interested in interventions that could reduce the illusion of argument justification and the influence of emotional investment and explored both influences that could be considered more tacit and those that more explicitly focused on taking other coherent perspective. Tacit influences can drastically influence a wide variety of tasks such as evaluating contingencies and reporting belief in God. (Tversky, & Kahneman, 1981; Shenhav, Rand, & Greene, 2012). In Study 2, we use tacit influences that both reframed the task and encouraged reflective thought. In Study 3, we investigate the impact of considering alternative perspectives on objective self-assessment. Unlike a typical explanation, arguments contain competing positions on the same issue, often creating multiple irreconcilable perspectives. Only considering one hypothesis leads to overconfidence. Explicitly instructing participants to be “objective and unbiased” does not eliminate this bias (Lord, Lepper & Preston, 1984). However, taking into account and articulating other possible hypotheses prevents overreliance on one explanation. (Gettys, Mehle & Fisher, 1986; Brem & Rips, 2000). Thus, a potential source of greater objectivity is “considering the opposite” (Lord, Lepper, Preston, 1984) and we therefore predict that the illusion of argument justification may only occur when people fail to actively engage alternative points of view. Although people may be adept at generating arguments and ignoring disconfirming evidence, the degree to which people overestimate their ability to justify through arguments has not been investigated. This error could be costly. When contemplating important issues, self-reflection may lead to a belief that one has an adequate grasp of the underlying arguments, when in fact one has only a superficial understanding. In an area like politics, where arguments can eventually lead to important public policy, it seems especially important to know how well an argument is understood. It is an open question whether we accurately assess the quality of arguments that we are about to make; we often engage in arguments and through experience we could become properly calibrated. However, based on the other metacognitive inaccuracies, we predict that there will be an illusion of justification: people will overestimate their ability to justify to others through arguments before actually having articulated them. Furthermore, we predict that deeper emotional involvement in an issue will correspond with a larger illusion of argument justification.