Understanding conflicting information on social science issues 1 Running Head: Understanding conflicting information on social science issues How non-experts understand conflicting information on social science issues: The role of perceived plausibility and reading goals

When people read Web-based science-related journalistic articles, they usually read more than one text. As a consequence, they face the task to construct a coherent mental model of the issue on the basis of conflicting information. We assume that recipients handle this task by evaluating the plausibility of information against their current understanding and prior knowledge (epistemic validation). On the one hand, information judged as implausible is often not processed further, yielding a bias towards plausible information in the mental model (plausibility bias). On the other hand, recipients can also engage in elaborative processing when they are motivated to develop a justified point of view (epistemic reading goal) which should lead to a rich mental model in contrast to the goal to memorize facts (receptive reading goal). The present study investigated the relationships of perceived plausibility and comprehension of multiple articles related to a social science topic (the PISA study) and effects of recipients’ reading goal with multilevel models (items nested within recipients) on a trial-by-trial basis. As predicted, information judged as plausible was more likely integrated into recipients’ mental model. This plausibility bias was independent of recipients’ reading goal. However, an epistemic reading goal led to an overall stronger mental model than a receptive reading goal. Moreover, we found a positive relationship between perceived plausibility and memory for text. The present results demonstrate that recipients make sense of science communication about controversial issues by actively monitoring the plausibility of information and regulating comprehension processes according to their reading goals. Reference Terms: comprehension, epistemic validation, plausibility, reading goal, science communication Understanding conflicting information on social science issues 3 Introduction The World Wide Web is one of the main sources for science-related information for lay persons and scientists alike. New scientific results are produced and published at a fast pace, often prompting an instant discussion on the Internet. As a consequence, the World Wide Web offers a multitude of documents which represent different perspectives, dissimilar empirical evidence and conflicting argumentative positions on the same scientific issue. Taken together, these circumstances provide new challenges for recipients of science-related media products on the Web. Getting an adequate picture about the validity of explanatory models of climate change, for instance, requires recipients to process divergent and often conflicting information from different web sites, to evaluate the credibility and plausibility of this information and to integrate it into a coherent and well-justified point of view (Perfetti et al., 1999; Rouet, 2006). In contrast to reading science textbooks, where relationships between different explanatory models of scientific findings are made explicit and divergent perspectives are usually resolved, using the World Wide Web as source of science-related information requires recipients to resolve inconsistencies from multiple web-based articles on their own. In this article, we will argue that recipients of science-related journal articles on the Web handle this problem by using their prior knowledge to judge the validity and plausibility of new information (epistemic validation, Richter, 2003; Richter, 2011), which often leads to a plausibility bias in the mental representation (Schroeder, Richter, & Hoever, 2008). Against this background, the present study investigated the relationship between plausibility judgments and comprehension outcomes constructed in studying multiple articles that contain conflicting information. Moreover, recipients search and read science-related information on the Web for different purposes and the processing of conflicting information from multiple texts is a partly strategic (goal-dependent) process (Richter & Schmid, 2010; Wiley & Voss, 1999). For these Understanding conflicting information on social science issues 4 reasons, we examined whether and to what extent recipients’ reading goal affects the relationship of plausibility judgments and comprehension outcomes. In the following sections, we will discuss these issues in turn. We will then report data from a web-based experiment in which university students read multiple journal articles on an issue from the social/educational sciences, which is hotly debated in public every three years when new results are published: what conclusions (if any) can be drawn from the results of the PISA study (OECD Programme for International Student Assessment; Klieme et al., 2010; OECD, 2010)? To investigate the relationship between perceived plausibility and comprehension on different levels of representation (memory for text and the situation model, Kintsch, 1988), two types of comprehension tasks (recognition and verification) were used with different types of test items. Relationships between responses to these tasks and plausibility judgments were analyzed with a multilevel design (test items nested within persons; Raudenbush & Bryk, 2002). This multilevel approach enabled us to examine on a trial-by-trial basis how the perceived plausibility of certain pieces of information is related to the integration of the same information into recipients’ mental representation of the controversial issue. Multiple Documents Comprehension and Epistemic Validation Reading multiple journalistic articles on science-related issues is an instance of multiple documents comprehension (Perfetti et al., 1999). Perfetti et al. (1999) proposed that in the best of cases recipients of multiple documents construct a rich documents model which consists of two parts: a referential representation of the text content attached to the respective source information and an intertext model that represents argumentative relationships – including conflicting and discrepant information – between different documents. This task requires recipients not only to comprehend, memorize and integrate information from various documents. In order to detect conflicting and discrepant information, recipients also need to evaluate the plausibility of Understanding conflicting information on social science issues 5 arguments and information presented in a given document in the light of arguments and information from other documents and their own knowledge and beliefs (epistemic validation, Richter, 2003). In previous research, the validation of text information has often been conceptualized as part of a broader metacognitive activity of comprehension monitoring which refers to evaluating one’s own level of understanding of texts during reading and the detection of comprehension difficulties (Westby, 2004). For example, Baker (1985, 1989) distinguishes in her theoretical account of metacomprehension between two higher-level comprehension monitoring standards that are closely related to epistemic validation: the external consistency and the internal consistency standard. Evaluating comprehension for external consistency means validating text information with regard to own prior knowledge, thus checking texts for violations of prior knowledge. Evaluating comprehension for internal consistency refers to the validation of text ideas in light of other text ideas, thus detecting contradictions and argumentative fallacies in the texts. Baker (1985) found in a study with college students that readers differ in their use of the internal and external consistency standards. Only readers with high verbal abilities consistently and spontaneously acknowledged violations of these standards, i.e., information that contradicted general world knowledge or other information given in a text. However, other studies using more indirect methods such as reading times, eye-tracking, or event-related potentials suggest that readers routinely monitor the plausibility and internal consistency of incoming textual information, even though they might not necessarily be able to report violations of the external and internal consistency standards (e.g., Baker, 1989; Lea, 1995; Lea, Mulligan, & Walton, 2005; Richter, Schroeder, & Wöhrmann, 2009; Singer, 1993, 2006; Singer, Halldorson, Lear, & Andrusiak, 1992). For example, readers have no problem in detecting logical inconsistencies in a text when the logical relationship is signaled or the premise Understanding conflicting information on social science issues 6 information is held active in working memory (Lea, 1995; Lea et al., 2005). Recipients also validate the inferred bridging information when they draw causal and other types of bridging inferences (Singer et al., 1992; Singer, 1993). Singer (2006) used a reading time paradigm with stories that contained target sentences (e.g., The coach determined/figured that it was oranges that Ken ate) that were either consistent or inconsistent with the situation established by a sentence provided earlier (e.g., On this day, it was very hot and Ken and his brother gobbled some oranges/apples). The pattern of reading times for these sentences demonstrated that inconsistent sentences produced longer reading times. Moreover, in an event-related potentials study (ERP) with the same materials, Feretti and colleagues found an extended N400 response that occurred in implausible sentences immediately after the word that created the implausibility (Ferretti, Singer, & Patterson, 2008). In ERP studies, the N400-response describes a stereotyped electrophysiological response that is associated with semantic processing. In particular, it may be regarded as an index for integration costs associated with information which does not immediately make sense given the linguistic context of a message. The extended N400 negativity response after the word that caused the implausibility most likely indicates pragmatic processing costs associated with the impl