On many topics, people disagree about the answers to factual questions, and their beliefs are often inaccurate in predictable directions. People have differing beliefs about questions related to income mobility, crime rates, and racial discrimination in labor markets; tend to be biased in the direction that is more representative of their political party's stances; and often overestimate their own political knowledge (e.g. Alesina, Stantcheva, and Teso 2018;Flynn, Nyhan, and Reifler 2017;Ortoleva and Snowberg 2015). As shown by Gerber and Huber (2009), Meeuwis et al. (2019), and Allcott et al. (2020), these partisan beliefs can affect consumer, financial, and public health behavior. Given the significance of these issues, why does such bias and belief polarization persist? After receiving a piece of news, people immediately form posterior beliefs in ways that incorporate both their prior beliefs and how they perceive the informativeness of the news. If we only observe beliefs at a snapshot in time, two people's disagreement can be consistent with many explanations: for instance, they may have started with different beliefs, they may receive news from different sources, or they may have differently-distorted inference processes. The first two channels are often relevant in politicized settings. First, Democrats and Republicans often have different priors, leading to differences in posteriors; this can be consistent with Bayes' rule. Second, Democrats and Republicans often consume different media, and may find arguments akin to those from MSNBC and from Fox News differentially informative. 1This paper introduces a new tool for detecting a third channel, motivated reasoning, which posits that people distort their inference process in the direction of states they find more attractive. In many settings, we may think that people are motivated to hold beliefs that align with their political party's stances: Republicans are motivated to believe that increasing immigration is associated with higher crime rates; Democrats are motivated to believe that there is severe racial discrimination in labor markets; and both parties are motivated to believe that members of their party are more knowledgeable about these issues than are members of the opposing party. When people receive news, they are often reminded of what beliefs they currently hold, and of beliefs they find more attractive to hold (motives). They then use these previously-hypothesized domains. The second challenge is that in many domains of interest, such as politics, people enter into experiments with preconceived beliefs. Eliciting motivated reasoning in such settings is difficult; researchers typically either restrict tests to questions that have binary answers (e.g. Mobius et al. 2022), ask numerous questions in order to elicit entire prior and posterior belief distributions (e.g. Eil and Rao 2011), or do not fully rule out Bayesian updating (e.g. Sunstein et al. 2017). My design is able to identify motivated reasoning from Bayesian updating on questions that have numerical answers, while only requiring elicitation of one moment of subjects' belief distribution: their median. Researchers can use this design to test how people motivatedly reason about essentially any factual question with a numerical answer. The design has two main steps. First, each subject is given a variety of factual questions with numerical answers. On each question, each subject selects a response that they think is equally likely to be above or below the correct answer; that is, the median of their belief distribution is elicited. Second, the subject is given one binary message that is chosen randomly from either a True News source or a Fake News source; the message tells her whether the answer was above or below her median. If the message is from True News, it is always accurate. If the message is from Fake News, it is always inaccurate. The subject is not told which source the message came from; instead, she is asked to make inferences about the source's veracity from the message she receives. Since messages relate the true answer to subjects' subjective median, a Bayesian (she) would believe that it is equally likely for each source to report either message. That is, she has stated that she believes the answer is equally likely to be greater than or less than her median; so, she believes the likelihood that a True News source would report that the answer is greater than her median is 1/2, and the likelihood that a Fake News source would report that the answer is less than her median is also equal to 1/2. Therefore, the Bayesian would find a "greater than" message to be completely uninformative about the veracity of the news source. Likewise, she would find a "less than" message to be uninformative. On the other hand, a subject who engages in motivated reasoning (he) will trust the news more if it sends a message that supports what he is more motivated to believe. If he engages in politically-motivated reasoning, he will assess messages that align with the beliefs of his political party (Pro-Party news) to be more truthful, Table 1: The list of topics and hypothesized motives in the experiment. while assessing messages that misalign (Anti-Party news) to be less truthful. I test for motivated reasoning using a within-subject experiment on Amazon Mechanical Turk with approximately 1,000 people in the United States. As a testament to the portability of the design, I run the test on nine different economic, political, and social questions (politicized topics), and on one question about own performance in the experiment. The list of topics and hypothesized motives is in Table 1. The main finding from the experiment is that Bayesian updating is strongly rejected in favor of politically-motivated reasoning on the politicized topics. While a Bayesian would believe that Pro-Party and Anti-Party news are equally likely to be True News on the politicized topics, subjects in the experiment believe that Pro-Party messages are 9 percentage points more likely than Anti-Party messages to come from the True News source. These effects are moderate but very precisely estimated (s.e. 0.6 percentage points; p ≈ 10 -40 ).
This test is precise enough to measure each topic individually; for eight of the nine politicized topics, the treatment effect is statistically significantly larger than zero (p ≤ 0.001). To my knowledge, this experiment provides the first evidence for motivated reasoning on these questions that is not confounded by Bayesian updating or prior-confirming biases (like those in Festinger 1957; Lord, Ross, and Lepper 1979;Tetlock 1983;and Rabin and Schrag 1999). 4 Treatment effects are larger for subjects who are more partisan. I also find consistent evidence that subjects motivatedly reason about their performance. The new design could lead to potential confounds that are unique to this experiment, such as if subjects systematically misreport their median belief or misinterpret the experiment's definitions of True News and Fake News, but I find that results are unlikely to be explained by these alternative hypotheses. 5 To account for unmotivated mistakes, I also compare behavior on political topics to three neutral topics, and find that news veracity assessments on neutral topics are lower than assessments on Pro-Party news and higher than assessments on Anti-Party news. The second main finding is that people's systematically-biased beliefs about these topics is related to their motivated beliefs (as also discussed by Eil and Rao 2011). Since people who motivatedly reason about an issue will systematically distort their beliefs, we can partly infer what people's motives are by looking at their current beliefs. That is, the direction of one's error predicts the direction of one's motive. In the context of the experiment, this hypothesis predicts that people will give higher veracity assessments to news that (falsely) reinforces their error compared to news that (truthfully) brings them closer to the correct answer. Indeed, subjects in the experiment trust the error-reinforcing Fake News more than the error-correcting True News, and only do so on topics where motivated reasoning is expected to play a role. I also show how motivated reasoning can lead to apparent overprecision, arguably the most durable form of overconfidence (Moore and Healy 2008;Moore, Tenney, and Haran 2015). Motivated reasoning may provide a link between overprecision and partisanship, a relationship documented in Ortoleva and Snowberg (2015), because partisans' belief distributions are more miscalibrated. Miscalibrations are more severe with stronger motives, and this leads 50-percent confidence intervals to contain the 4 Papers that find asymmetric responses to information on related topics include: Taber and Lodge (2006) [gun laws]; Alesina, Stantcheva, and Teso (2018) [upward mobility]; Cappelen, Haaland, and Tungodden (2018) [responses to taxation]; Haaland and Roth (2019) [racial labor market discrimination]; Sarsons (2017) and Kunda and Sinclair (2000) [gender and performance]; Alesina, Miano, and Stantcheva (2022), Haaland andRoth (2020), andDruckman, Peterson, andSlothuus (2013) [immigration]; Nyhan andReifler (2013) andNyhan, Reifler, andUbel (2013) [perceptions of elected officials]; and Sunstein et al. (2017) [climate change]. Many findings from these papers may be due to motivated reasoning. 5 As described in Section 4.4 and appendices, predictions are identical if subjects mistakenly believe Fake News sends random messages instead of always-false messages, and results are not driven by subjects who have skewed belief distributions and misreport their median. true answer less than 50 percent of the time, especially for partisans. Motivated reasoning not only affects how people trust or distrust news, but also impacts how people change their beliefs about the topics themselves. Despite being uninformative, the messages lead to further belief polarization: subjects are more likely to revise their beliefs away from the population mean than towards it. That is, informational content is not a necessary condition for belief polarization. 6 Politicallymotivated reasoning helps reconcile the notions that the ideological polarization of beliefs may be high, even if the ideological polarization of information acquisition is modest (Gentzkow and Shapiro 2011). 7 It can also explain why news providers may distort messages to appeal to their audience's motivated beliefs (Thaler 2022). There are no sizable demographic heterogeneities in politically-motivated reasoning, neither in direction nor magnitude, once party preference is controlled for. Differences in treatment effects are almost all statistically indistinguishable from zero, and estimates are precise enough to rule out modest effect sizes. 8 Motivated beliefs on these questions seem to be principally driven by politics. Lastly, to complement the experiment, I discuss a model of motivated reasoning in which people distort their inference when they receive new information. This model adds to the decades-old theoretical literature (such as Akerlof and Dickens 1982;Carrillo and Mariotti 2000;and Benabou and Tirole 2002) by emphasizing the non-strategic nature of the bias: agents in this model do not distort their beliefs for functional reasons. 9 They make inferences using a modified Bayes' rule, weighting priors and likelihoods as a Bayesian would, but act as if they receive an extra signal that puts more weight on higher-motive states. 10 This leads them to make inferences 6 Linking this finding to the polarization in trust in news relates to a growing literature that discusses the relationship between trust in news and political partisanship (Nisbet, Cooper, and Garrett 2015;Levendusky 2013;Druckman, Levendusky, and McLain 2018). 7 Gentzkow and Shapiro (2006) and Gentzkow, Wong, and Zhang (2018) provide alternative theoretical explanations with Bayesian agents who have different priors, but these models do not predict updating from uninformative signals. 8 On the other hand, Thaler (2021) shows that performance-motivated reasoning has substantial heterogeneity by gender; only men systematically motivatedly reason to think their performance was higher, while women are Bayesian on average. 9 This directly contrasts the model with optimal beliefs models such as Brunnermeier and Parker (2005) and Mobius et al. (2022). It is more similar to the selective memory models of Benabou and Tirole (2002) and Benabou and Tirole (2011) and the Bayesian model of Mayraz (2019), but emphasizes distorting the processing of new information. 10 This is a similar theory as Kahan (2016), but differs from related economic models of motivated cognition like Benabou andTirole (2011) andMobius et al. (2022), in that it assumes directional distortions but not underreaction to information. even when a Bayesian does not.
To further ensure that experimental results were unlikely to be driven by noise, I retested the main hypotheses using a preregistered replication one year later on a new sample. The findings successfully replicated; for details, see Online Appendix D.

The rest of the paper proceeds as follows: Section 2 develops the model of motivated reasoning. Section 3 presents the experimental design and main hypotheses. Section 4 analyzes experimental results, including the evidence for motivated reasoning, treatment effect heterogeneity, and overprecision. Section 5 concludes and proposes directions for future work. This section outlines the model of motivated reasoning. Agents distort how they process new information by acting as if they receive an additional signal corresponding to their motivated beliefs. When signals are uninformative to a Bayesian, as will be the case in the experiment, this distortion can lead motivated reasoners to update in the direction that they find more attractive. There are a set of agents i and a set of questions q. For each question, Nature draws a state θ q from Θ q ∈ R. For each agent and question, Nature draws a source from S: the source is either True (T iq ) or Fake (F iq ). Sources are independently drawn with P (T iq ) = p, and agents receive data x iq about the source veracity. Agents engage in motivated reasoning, which means that they form their posterior by incorporating prior, likelihood, and a motivated beliefs term:

We take log odds ratios of both sides (Grether 1980) to attain an additive form:

Motivated reasoners act as if they receive both the actual signal (x iq ) and a signal whose relative likelihood corresponds to how much they are motivated to believe the state is T iq given x iq . m iq : S → R is called the motive function. The weight put on this signal is ϕ ≥ 0, called susceptibility. ϕ = 0 corresponds to Bayes' rule; ϕ > 0 corresponds to motivated reasoning. I now introduce an information structure in which the x iq are binary and uninformative about the news source veracity, but affect m iq . The x iq are chosen such that P(x iq |T iq ) = P(x iq |F iq ) = 1/2. In such a setting, described below, a Bayesian will set P(T iq |x iq ) = p, but a motivated reasoner will set logit P(T iq |x iq ) = logit p + ϕ(m iq (T iq , x iq ) -m iq (F iq , x iq )). That is, if the motivated reasoner finds x iq more attractive, then they will form a posterior that the message comes from a truthful source. Meanwhile, a Bayesian will not update from either message because the message is uninformative about the source. In many environments, beliefs about P(x iq |T iq ) relate to m iq (T iq , x iq ) since agents who are motivated to believe x iq is true may currently believe that x iq is true. As such, it is necessary to construct a signal structure that tailors messages to agents' current beliefs. For each q, agents report a median belief µ iq such that they believe θ q is equally likely to be greater than µ iq or less than µ iq .12 Nature then chooses one of two news sources from S. The news source sends a message to the agent that either says that θ q > µ iq or says that θ q < µ iq : x iq (µ iq ) ∈ {G(µ iq ), L(µ iq )}. The True source always sends the "truthful" message and the Fake source always sends the "untruthful" message:

After receiving x iq (µ iq ), agents report how likely they think it is that the message came from the True source: a(x iq ; µ iq ). They also report a revised median belief µ iq (x iq ; µ iq ). Finally, the states θ q and sources are revealed. Agents may either be motivated to believe θ q is higher than µ iq or θ q is lower than µ iq . An agent with a high-θ q motive has m iq (T iq , G(µ iq )) = m iq (F iq , L(µ iq )) > m iq (T iq , L(µ iq )) = m iq (F iq , G(µ iq )), and vice versa for an agent with a low-θ q motive. We now consider how a Bayesian and a motivated reasoner update their beliefs about the news source. Given message G(µ iq ), the Bayesian updates according to Equation (1) with ϕ = 0:

Since the Bayesian has stated her median belief µ iq , she thinks that both messages are equally likely ex ante and does not update in either direction. Meanwhile, the motivated reasoner updates according to Equation (1) with ϕ > 0:

logit a(G(µ iq )) = logit P(T (µ iq )) + log P(G(µ iq )|T (µ iq ))

That is, we have the following:

The procedure above identifies motivated reasoning from Bayesian updating:

• Motivated reasoning (ϕ > 0):

This design identifies whether agents are more motivated to believe that the true state is above their median belief µ iq or to believe that the true state is below µ iq . More precisely, this design provides a joint test of ϕ > 0 and the direction of mo-tives. In the political context, we will consider agents who are either of a Republican type or of a Democratic type, assume that m Rq (S Rq , x(µ Rq )) = -m Dq (S Dq , x(µ Dq )), and then test whether, given these motives, ϕ > 0. Since signals are uninformative, this design also identifies motivated reasoning from several other non-Bayesian, models of inference.
Consider the following generalized class of updating rules:

This class of updating rules includes a prior-confirming bias (ζ > 1), conservatism (κ < 1), and base-rate neglect (ζ < 1). In all these cases, agents form the same posterior after G(µ iq ) and L(µ iq ) (even if κ differs for good and bad news). In addition, by taking Θ q as the domain for the motive function, the same logic as in Section 2.1 leads motivated reasoners to be more likely to revise their median beliefs in the higher-motive direction. A Bayesian will have P µ iq (x iq ) > µ iq = P µ iq (x iq ) < µ iq . However, a motivated reasoner who has m iq (θ q ) increasing in θ q will be more likely to update upwards: P µ iq (x iq ) > µ iq > P µ iq (x iq ) < µ iq , and vice versa if m iq (θ q ) is decreasing in θ q . Online Appendix C discusses extensions to the model that can generate further predictions. Agents who start out believing that µ iq is larger, due to receiving signals prior to this game, will be motivated to believe that θ q is even larger than µ iq , as current beliefs are reflective of motivated beliefs. With additional functional form assumptions, motivated reasoning also leads to overprecision and overconfidence. This section details the experiment introduced above. Section 3.1 outlines the experimental design, Section 3.2 discusses specific pages that subjects see, and Section 3.3 generates the main hypotheses. Section 3.4 discusses the participant sample and further details of the data collected. Specific question wordings and screenshots for every page type, including instructions and scoring rules, are in the Online Appendices. To fix ideas, consider the following question, taken verbatim from the experiment: Some people believe that the Obama administration was too soft on crime and that violent crime increased during his presidency, while others believe that President Obama's pushes towards criminal justice reform and reducing incarceration did not increase violent crime. This question asks how murder and manslaughter rates changed during the Obama administration. In 2008 (before Obama became president), the murder and manslaughter rate was 54 per million Americans. In 2016 (at the end of Obama's presidency), what was the per-million murder and manslaughter rate? This is a question for which we will hypothesize that Republicans are motivated to believe the answer is higher, and Democrats are motivated to believe the answer is lower. The test of motivated reasoning involves the following steps:

1. Beliefs: Subjects are asked to guess the answers to politicized questions like the one above. They are asked and incentivized to guess their median belief (such that they find it equally likely for the answer to be above or below their guess). They are also asked and incentivized for their interquartile range. Details on incentives are below. 2. News: On each question subjects receive one binary message from one of two randomly-chosen news sources: True News and Fake News. The message from True News is always correct, and the message from Fake News is always incorrect. This is the main (within-subject) treatment variation. The message says either "The answer is greater than your previous guess of [previous guess]. " or "The answer is less than your previous guess of [previous guess]." Note that the exact messages are different for subjects who make different initial guesses. For the question above, "greater than" corresponds to Pro-Republican News and "less than" to Pro-Democratic News. For subjects who support the Republi-can Party more than the Democratic Party, "greater than" is coded as Pro-Party news and "less than" is coded as Anti-Party news, and vice versa for subjects who support the Democratic Party more. 3. Assessment: After receiving the message, subjects assess the probability that the message came from True News using a scale from 0/10 to 10/10, and are incentivized to state their true belief. I test for motivated reasoning by looking at the treatment effect of variation in news direction on news veracity assessments. Recall that since subjects receive messages that compare the answer to their median, a Bayesian would not change her assessment based on the message. If she had a prior that P(True News) = 1/2 before seeing the message, she would form a posterior that P(True | "greater than") = P(True | "less than") = 1/2. We attribute systematic treatment effects of the messages on assessments to motivated reasoning. For instance, if Republicans tend to state P(True | "greater than") > P(True | "less than") and Democrats tend to state P(True | "greater than") < P(True | "less than") on the question above, this would be coded as politically-motivated reasoning. Subjects see 14 questions in the experiment, including the ten topics in Table 1, three neutral questions, and one comprehension check. 13 The timing is as follows: Demographics Question 1 News 1 ...

News 14

The Demographics page includes questions about party ratings, party affiliation, ideology, gender, age, race and ethnicity, annual income, highest education level, state or territory of residence, and religion. Party ratings are used to categorize subjects' party preferences; subjects are asked to rate the Democratic and Republican parties using a 0-100 scale that is akin to the feeling thermometer used in the American National Election Studies. The Results page tells subjects what their overall performance was, what their score on each question and assessment was, and the correct answer to each question and assessment. Subjects are told that they will see this page at the beginning of the experiment, and they are forced to go through it before exiting the study and receiving payment. Forcing subjects to see the answers aims to limit subjects' scope for strategic self-deception. 14  The order of Questions 1-12 is randomized between subjects, but Questions 13 and 14 are fixed. 15 Question 13 asks subjects how they scored on Questions 1-12 relative to 100 pilot subjects.
Question 14 asks subjects to compare Democratic pilot subjects' performance to Republican pilot subjects' performance on Questions 1-12. 16

At the end of the experiment, subjects earn a show-up fee of $3 and either receive an additional $10 bonus or no additional bonus. As described below, in each round of the experiment subjects earn between 0-100 "points" based on their performance. These points correspond to the probability that the subject wins the bonus: a score of x points corresponds to an x/10 percent chance of winning the bonus. 17

On question pages, subjects are given the round number, the topic, the text of the question, and are asked to input three numbers about their initial beliefs:

• My Guess: This elicits the median of subjects' prior distribution. • My Lower Bound: This elicits the 25th percentile of subjects' prior distribution. • My Upper Bound: This elicits the 75th percentile of subjects' prior distribution. The scoring rule for guesses is piecewise linear. Subjects earn max{100-|c-g|, 0} points for a guess of g when the correct answer is c. Subjects are told that they will maximize expected points by stating the median of their belief distribution. 14 Subjects spend 71 seconds on the results page on average, suggesting that they are reading it. 15 Main questions are equally likely to be selected in each round, but the comprehension check is restricted to be between Question 2-11. This restriction ensures that subjects still pay attention after the first question, and to make sure that a willingness-to-pay round, which occurs for Question 12, does not overlap with the comprehension check. 16 Half of subjects are given the Democrats' score and asked to predict the Republicans'; half are given the Republicans' score and asked to predict the Democrats'. 17 This earnings system is similar to the most broadly incentive-compatible one from Azrieli, Chambers, and Healy (2018) in which subjects are paid randomly for one round. I use my procedure instead in order to allow for a clearer measure of "performance" that is used as a question in the experiment. I do not need to assume risk neutrality in order for the experiment to be incentive compatible, but I do need to assume linearity in probabilities. The scoring rule for bounds is piecewise linear with different slopes. For upper bound ub, subjects earn max{100 -3(c -ub), 0} points if c ≥ ub and max{100 -(ubc), 0} points if c ≤ ub. For lower bound lb, subjects earn max{100 -(c -lb), 0} points if c ≥ lb and max{100 -3(lb -c), 0} points if c ≤ lb. Subjects maximize expected points by setting ub to be the 75 th percentile and lb to be the 25 th percentile of their belief distribution. Subjects are restricted to give answers for which My Lower Bound ≤ My Guess ≤ My Upper Bound; if they do not, they see an error message. After submitting My Guess, subjects see a second page about the same question. At the top of the page is the text of the original question. Below the question, there is a message relating the correct answer to the number they submitted for My Guess. This message says either: "The answer is greater than your previous guess of [My Guess]." or "The answer is less than your previous guess of [My Guess]." Subjects are told that True News always tells the truth and Fake News never tells the truth, and that sources are independent across questions. Below the message, subjects are asked: "Do you think this information is from True News or Fake News?" and choose one of eleven radio buttons that say "x/10 chance it's True News, (10x)/10 chance it's Fake News" from each x=0, 1, . . . , 10. The scoring rule for assessments is quadratic. For assessment a, subjects earn 100(1 -(1 -a) 2 ) points if the source is True News and 100(1 -a 2 ) points if it is Fake News. Subjects maximize expected points by stating the closest multiple of 0.1 to their belief. They are given a table with the points earned as a function of each assessment and news type. Occasionally, a subject will correctly guess the answer. If this happens, she skips the news assessment page and moves on to the next question. 18Half of subjects also see a "Second Guess" part of the News Assessment page. For these subjects, below each news assessment question they are asked an additional question: "After seeing this message and assessing its truthfulness, what is your guess of the answer to the original question?" Subjects are given the same linear scoring rule as on the initial guess. They earn max{100 -|c -g|, 0} points for a guess of g when the correct answer is c. These second guesses will be used as robustness, as well as to test for belief polarization about how subjects change their guesses. The other half of subjects see an additional page between Question 12 and News 12, on which they are given instructions and asked to submit a willingness-to-pay (WTP) for a message. Due to space constraints, this group is not discussed in the main text. For instructions and results, see the Online Appendix. 19I randomly vary whether subjects are given a prior at the beginning of the experiment that P(True News) = 0.5 or not given any prior. One-third of subjects are given a prior. This randomization serves as a robustness check to ensure that results are not entirely driven by either updating over meta-priors over the probability of True News (as may be the case for the group who is not given a prior) or by anchoring to a 50-50 benchmark (as may be the case for the group who is given a prior). I find no statistically significant differences between the two groups on any main outcomes, so results in the main text of this paper pool all subjects.
Results are robust to restricting the sample to individual groups, as shown in the Online Appendix. The main hypothesis is that a news veracity assessment will be larger when it leads to a higher motive. This is a joint test that (1) people motivatedly reason, giving higher assessments to news in the direction of higher motives than to news in the direction of lower motives, and (2) the predicted direction of motives is as in Table 1. News on neutral topics is assumed to not affect motives. For politicized topics, the level of partisanship affects the steepness of the motive function. • Assessments of Pro-Party news are greater than assessments of Anti-Party news. • Assessments of Neutral topic news lie in between those of Anti-Party news and those of Pro-Party news. • The gap in assessments of Pro-Party news and Anti-Party news increases in partisanship. The hypothesis for Pro-Performance news and Anti-Performance news is similar. The next hypothesis is based on an observation that subjects may have different beliefs because these beliefs reflect past instances of motivated reasoning. When motives are unobservable, an experimenter can learn about subjects' motives by looking at their initial beliefs. Two subjects who motivatedly reason in different directions will hold different median beliefs: A motivated reasoner with an increasing motive function will be more likely to hold a median belief that is too high, and a motivated reasoner with a decreasing motive function will be more likely to hold a median belief that is too low. Therefore, a subject whose median is too high is more likely to have an increasing motive function as compared to a subject whose median is too low. If the two subjects make news assessments using the procedure above, they will trust news that reinforces the error in their initial beliefs more than news that mitigates the error. This behavior occurs even though signals are designed so that their interpretation is distinct from their beliefs. In the experiment, this behavior implies that subjects will give higher assessments to error-reinforcing news compared to error-mitigating news. Recalling that error-reinforcing news is Fake News and error-mitigating news is True News, we have the following:

Hypothesis 2 (Motivated reasoning and trust in Fake News)

• Assessments of Fake News are greater than assessments of True News on politicized topics, but not on neutral topics. • Assessments of Fake News are greater than assessments of True News, controlling for whether the news is Pro-Party or Anti-Party. We can use the second-guess group to construct an alternative test of motivated reasoning. By comparing subjects' first and second guesses, we can retest the main politically-motivated reasoning prediction and also study a form of belief polarization. First, as in Hypothesis 1, subjects are expected to more frequently adjust their guesses in the direction that favors their preferred party. Second, by a similar logic to Hypothesis 2, motivated reasoning would lead subjects to be more likely to adjust their guesses away from the population mean than towards it. We define Follow Message as the ternary variable that takes value:

• 1 if the message says G and the second guess is greater than µ, or if the message says L and the second guess is less than µ;

• 0 if the second guess equals µ; and

• -1 if the message says G and the second guess is less than µ, or if the message says L and the second guess is greater than µ.

Polarizing says G if µ is greater than the population mean guess or L if µ is less than the population mean guess. Anti-Polarizing news says the opposite. Hypothesis 3 (Motivated reasoning and second guesses)

• Follow Message is larger for Pro-Party than for Anti-Party news. • Follow Message is larger for Polarizing than for Anti-Polarizing news. The experiment was conducted in June 2018 on Amazon's Mechanical Turk (MTurk) platform. MTurk is an online labor marketplace in which participants choose "Human Intelligence Tasks" to complete. MTurk has become a popular way to run economic experiments (e.g. Horton, Rand, and Zeckhauser 2011;Kuziemko et al. 2015), and Levay, Freese, and Druckman (2016) find that participants tend to have more diverse demographics than students in university laboratories with respect to politics. The experiment was coded using oTree, an open-source software based on the Django web application framework developed by Chen, Schonger, and Wickens (2016). The study was offered to MTurk workers currently living in the United States. 1,387 subjects were recruited and answered at least one question, and 1,300 subjects completed the study. Of these subjects, 987 (76 percent) passed simple attention and comprehension checks, and the rest are dropped from the analyses. 20  When asked for their party ratings, 627 subjects (64 percent) give a higher rating to the Democratic Party; 270 (27 percent) give a higher rating to the Republican Party; and 90 (9 percent) give the same rating to each party. 21 These subjects are labeled as "Pro-Dem," "Pro-Rep," and "Neutral," respectively, and for most analyses I drop the Neutral subjects. 2/3 of subjects were randomly assigned to not receive a prior about the veracity of the news source, and 1/3 of subjects were told that True News and Fake News 20 The Online Appendix shows that main results are robust to inclusion of these subjects. In order to pass these checks, subjects needed to perfectly answer the attention check question in Appendix A by giving a correct answer, correct bounds, and answering the news assessment with certainty. In addition, some questions had maximum and minimum possible answers (e.g. percentages, between 0 and 100). Subjects were dropped if any of their answers did not lie within these bounds. 21 Levay, Freese, and Druckman ( 2016) also find that subjects on MTurk are mostly Democratic. were equally likely. Independently, 1/2 of subjects were randomly assigned to receive a WTP question and 1/2 were asked for their second guesses.
Each subject answered 13 questions; there are a total of 11,661 guesses to questions for the 897 non-neutral subjects. There are 11,443 news assessments. The discrepancy between these numbers is due to 143 subjects in the WTP group who did not receive a message in one round, and due to 75 (0.7 percent) correct guesses.22 I drop these observations for news assessments, leaving 7,902 observations on politicized topics, 891 on the performance topic, and 2,650 on neutral topics. The balance table for the Pro-Party / Anti-Party treatment is in Table 5. The treatment is well-balanced, and overall shares of Pro-Party and Anti-Party news are nearly identical, suggesting that there was no differential attrition by treatment. In order to validate the main results, I ran a replication experiment one year later. As discussed in the Online Appendix, the replication confirms the results.23

This subsection shows that the raw data support the main hypotheses, and the following subsection shows the relevant regressions. To validate that these questions are politicized, Appendix Table 4 compares initial guesses and finds systematic differences in median beliefs by party consistent with the directions predicted in Table 1. In support of the first part of Hypothesis 1, I show that subjects trust the Pro-Party news more than they trust the Anti-Party news. Subjects' average assessment that the likelihood of Pro-Party news is from the True News source is 9.1 percentage points higher than their average assessment that Anti-Party news is from the True News source (s.e. 0.6 pp; p < 10 -40 ). Figure 1 shows the CDF of assessments for Pro-Party and Anti-Party news; the Pro-Party distribution first-order stochastically dominates the Anti-Party distribution. Appendix Figure 6 shows the same result for Pro-Performance and Anti-Performance news assessments. Figure 2 shows the subject-demeaned assessments by news direction (Pro-Party; Anti-Party; news on neutral topics) and subject type (Partisan and Moderate, as defined by the absolute difference in party ratings). In support of all three parts of Hypothesis 1, subjects believe that Pro-Party news is more likely than news on neutral topics to be true, believe that Anti-Party news is less likely than news on neutral topics to be true, and differences are larger for partisans. If neutral topics account for unmotivated mistakes in news assessments, then this figure shows that there is motivated reasoning in both positive and negative directions. In support of Hypothesis 2, subjects trust the Fake News more than they trust the True News. Subjects' average assessment that the likelihood of True News is actually from the True News source is 6.0 percentage points lower than their average assessment that Fake News is from the True News source (s.e. 0.6 pp; p < 10 -20 ). Appendix Figure 7 shows the CDF of assessments for True News and Fake News on the political questions; the Fake News distribution first-order stochastically dominates the True News distribution. This effect is not solely driven by whether the news is Pro-Party versus Anti-Party. Figure 3 shows the subject-demeaned assessments by news direction (Pro-Party; Anti-Party; Neutral) and news veracity (True News; Fake News). Subjects give higher assessments to Fake News than to True News on politicized topics, but they do not do so on neutral topics.24

The primary regression specification is within subject. 25 It regresses assessments a on Pro-Party news for subject i, question topic q, and round r with fixed effects for i, q, and r, restricting to news that is Pro-Party or Anti-Party:26

This specification is used in Table 2, column 2. A similar alternative specification, replacing individual-level fixed effects with demographic controls, is in column 1. Hypothesis 1 claims that the Pro-Party / Anti-Party gap is increasing in partisanship, so column 3 interacts partisanship (the absolute difference in party ratings) with Pro-Party news. It also claims that motivated reasoning leads to both higher assessments for Pro-Party news and lower assessments for Anti-Party news; as such, column 4 includes indicators for both Pro-Party (vs. Neutral) news and Anti-Party (vs. Neutral) news. Hypothesis 2 posits that subjects will trust Fake News more than True News on politicized topics, so columns 5 and 6 regress assessments on a dummy for True News, controlling for and not controlling for Pro-Party news. Hypotheses 1 and 2 are strongly supported. Assessments for Pro-Party news are higher than for Anti-Party news, and this effect increases in partisanship. There is evidence for motivated reasoning both towards Pro-Party and away from Anti-Party news, and Fake News assessments are higher than True News assessments. Next, we look at each topic individually by regressing on the interaction of treatment and topic dummies. Figure 4 shows evidence for politically-motivated reasoning on eight of the nine hypothesized topics (p < 0.001 on each). It also shows that people motivatedly reason towards believing they outperformed others (p < 0.001). To further validate the "trust in news" measure of motivated reasoning, I also show that subjects engage in asymmetric updating about the questions themselves. Half of subjects give a second guess to the initial question after receiving news, and I hypothesize that subjects are more likely to update in the Pro-Party direction than in the Anti-Party direction. This test is both useful as a robustness check as well as useful to better understand belief polarization on these questions. Column 1 of Table 3 shows that subjects are more likely to update their median in the direction of Pro-Party messages than they are from Anti-Party messages. Column 2 shows that on politicized topics, subjects are also more likely to change their guesses in the direction of a Polarizing message (one that tells them their guess is farther away from the mean) than from an Anti-Polarizing message. Only Pro-Party / Anti-Party news observations, as defined in Table 1. Polarizing News is news that tells subjects that, compared to their initial guess, the answer is in the opposite direction from the population mean.
Dependent variable is 1 if subjects change their guess upwards when the message says "Greater Than" or downwards when the message says "Less Than," -1 if they change their guess in the opposite direction, and 0 if they do not change their guess. Columns 4-6 of Table 3 show that discrepancies in both motivated posteriors and belief polarization can be explained by differences in news assessments. After controlling for assessments, guess changes are not statistically significantly affected by Pro-Party / Anti-Party messages, nor are they statistically significantly affected by polarizing messages, and the point estimates are close to zero. Evidence suggests that subjects change their beliefs to follow Pro-Party news more exactly because they trust that news source more. More broadly, these results give a stark prediction about how people change their beliefs. They show that, in environments where signals remind people of their motivated beliefs, informational content is not a necessary condition for belief polarization. There are features in this design that may lead subjects to behave in ways that are consistent with motivated reasoning but are also consistent with unmotivated confounding hypotheses. This subsection discusses a number of potential confounds, and shows how it is possible to test these alternative hypotheses. I argue that it is unlikely that these confounds can fully explain the results above, strengthening the interpretation that motivated reasoning is what is being identified in the experiment. It is reasonable to expect that subjects do not fully understand the concept of a median. For instance, they may answer with their mean belief instead. This would not directionally impact the news assessment results in a systematic direction, unless the prior distribution were notably skewed. We can use where the initial guess µ q lies in subjects' confidence intervals as a proxy for skewness, and see that the main results hold for subjects whose distributions are unskewed. On the politicized questions, 32 percent of subjects' guesses are exactly halfway between their upper and lower bounds. Table 6 uses the same specification as the main regression but interacts Pro-Party news, Anti-Party news, and True News with a dummy for having "unskewed" priors. The treatment effects are qualitatively and quantitatively similar, indicating that skewness does not directionally affect results. We have so far assumed that subjects treat news sources as being drawn from independent distributions. While subjects are explicitly told to do this in the instructions, it is useful to show that they are not using previous pieces of news to update about current pieces of news. In Table 7, I modify the main regression table to account for the relative number of Pro-and Anti-Party news in previous rounds. The effect of previous rounds' Proand Anti-Party news have no effect on the current round's assessment, and the main treatment effects are unchanged, suggesting that subjects indeed treat news sources as independent. Using the terminology "Fake News" can lead to greater subject engagement, but may also evoke other commonly-used definitions (such as fake articles, as in Allcott and Gentzkow 2017). This subsection shows that results cannot be explained by two plausible misinterpretations of Fake News. First, suppose that subjects believe that messages from Fake News are equally likely to send true and false messages, instead of always sending false messages. In this experiment, no predictions about assessments would change. A Bayesian would still have an ex-ante prior that Pro-Party and Anti-Party messages are equally likely, and would not infer anything about P(True) given either message. A motivated reasoner who is motivated to believe that the answer is large would still infer that P(True | Pro-Party) > P(True | Anti-Party). The same argument can be made if subjects believe Fake News randomly sends false messages with any interior probability. A more complicated situation involves subjects who believe that messages from Fake News are actually from a news source that is biased against their party. That is, suppose that subjects believe that Fake News is more likely to report Anti-Party news given a Pro-Party truth than Pro-Party news given an Anti-Party truth. To test this, we can again look at how subjects change their guesses. In particular, suppose that subjects were Bayesian but used this asymmetrically wrong definition of Fake News. Then, they would find Pro-Party "Fake News" messages to be more informative than Anti-Party "Fake News" messages, since "Fake News" is expected to usually send the Anti-Party message. (The quotes here indicate that these subjects are using the wrong definition.) Such subjects would then update more from Pro-Party than Anti-Party news, conditional on their assessment of P(True News). In Table 3, we see that subjects are similarly likely to update from Pro-Party and Anti-Party news after controlling for their assessments. While the data are too imprecise to rule out the existence of subjects who treat Fake News as biased, this story is insufficient for explaining the main effects. While it can sometimes be in subjects' best interests to strategically misreport their median in order to earn more points on news assessment questions, I find no evidence that subjects do this. 27 In Round 1 of the experiment, subjects do not yet know that they will be seeing a news assessment page. If subjects were strategically misguessing to earn more assessment points, they would perform worse in Round 1 than in subsequent rounds on assessments and better in Round 1 than in subsequent rounds on guesses. There are no statistically significant differences in assessment scores in Round 1. Subjects score 67.2 points (s.e. 0.9) in Round 1 and 66.4 points (s.e. 0.3) in Rounds 2-12; the difference is 0.8 points (s.e. 1.0; p = 0.383). 28 There are also no statistically significant differences in guess scores in Round 1. Subjects score 76.2 points (s.e. 1.0) in Round 1 and 75.9 points (s.e. 0.2) in Rounds 2-12; the difference is 0.3 points (s.e. 1.0; p = 0.758). Non-strategic forms of incorrect initial guesses are more complicated to rule out.
If there is symmetric noise in guesses, such that the probability that a subject is equally likely to state her Q quantile and her 1 -Q quantile for Q = 1/2, then the main results do not change. Results are also not consistent with subjects biasing their initial guesses towards the population mean. While such behavior can explain why subjects trust error-reinforcing news more than error-mitigating news on politicized, and why they trust Pro-Party news more than Anti-Party news, it incorrectly predicts the same pattern on neutral topics. The one form of misreporting that can be consistent with both Bayesian updating and results from the experiment involves subjects systematically misreporting initial guesses in a way that is biased in the opposite direction from their party. One potential reason for such a bias is that subjects do not sufficiently think about the question; and, given more time, they update towards their actual (more Pro-Party) belief. It is possible that seeing the second screen causes subjects to think harder about the original question, and thinking harder leads to more Pro-Party beliefs. 29 However, thinking harder does not seem to be asymmetric, as I find no evidence that subjects 27 For instance, if a subject believes that θ ∼ U [0, 20], then she will expect to earn more points on the news assessment question by guessing 20 than by guessing 10 since she is confident the correct answer is not greater than 20. She would expect to earn 90 points on the question and 100 points on news, as opposed to earning 95 points on the question and 75 points on news. 28 I exclude scoring on Rounds 13-14 since the questions are not randomly assigned in those rounds; the result is similar if they are included. I also exclude scoring on comprehension check questions. 29 The psychology behind this explanation overlaps with this theory of motivated reasoning, as the second page evokes the motive, and further work could better elucidate what qualifies as a signal. prefer to spend more time thinking about good news.30  Bursztyn et al. (2020) provides recent evidence showing that people in experiments may forgo payment in order to make political statements. In this experiment, if subjects have a preference for stating Pro-Party signals, then both their initial guesses and their news assessments will be biased in the Pro-Party direction, consistent with the data. However, if they are Bayesian, how they change their guesses will not be directional, since they have already stated their preferred belief. In Table 3, subjects are more likely to update their guesses in the Pro-Party direction than in the Anti-Party direction, even though they are equally likely to receive Pro-Party and Anti-Party news. This is consistent with subjects sincerely trusting the Pro-Party news more; it is not consistent with expressive Bayesian updating. As discussed in Section 3.4, subjects either were told that P(True News) = 1/2 in the instructions or were not told this. In the Online Appendix, I restrict the regressions from Table 2 to subjects in each randomization arm, and find similar results in the two arms. Likewise, seeing the WTP page or having a second guess has little effect. It is possible that subjects learn over the course of the experiment that they motivatedly reason and debias themselves. I interact the main effect with dummies for each round number, and do not find evidence for this. In each round, subjects give larger assessments to Pro-Party than to Anti-Party news. There are two types of heterogeneity to consider: heterogeneity in the direction of motivated reasoning, and heterogeneity in its magnitude. First, we consider the direction of heterogeneity. To do this, Figure 5 shows the coefficients from the regression of news assessments on the interaction of the political direction of the news (Pro-Rep vs. Pro-Dem) and party preferences, as well as on binarized observable demographics. Non-political demographics are race, gender, income, age, education, whether the subject's state voted for Trump or Clinton in 2016, and religious affiliation. Religious: subject affiliates with any religion. After controlling for party preference, none of the other demographics have a statistically significant effect on the direction of motivated reasoning. All coefficients are between plus and minus 0.03, a magnitude less than one-third than for politics. In the Online Appendix, we consider the magnitude of motivated reasoning, acknowledging that this design is unable to disentangle magnitude of bias and strength of motive. Partisans have a larger treatment effect than moderates, but demographics besides partisanship do not notably affect the magnitude of the bias; all effects are between +/-0.03. 31These results suggest that the magnitude of bias of motivated reasoning is broadly similar across demographics and that while the direction of motivated beliefs is heterogeneous by party, it is not very different across non-political demographics. We now consider two other relationships between initial beliefs and motivated reasoning: how much motivated reasoning can explain people's differences in beliefs, and how the bias relates to overprecise confidence intervals. First, the data show that variation in this experiment's measure of motivated beliefs can explain a sizable fraction of variation in actual beliefs about these questions. I look at the relationship between motives and beliefs by correlating answers to politicized questions with differences in assessments between Pro-Rep and Pro-Dem news. For each politicized question, subjects' initial guesses are winsorized (at the 5-percent level), normalized, and signed; positive numbers correspond to more Pro-Rep. Next, for each subject, these normalized guesses are averaged (and re-normalized) to give a measure of how Pro-Rep her beliefs are. I correlate this value with the normalized average difference between Pro-Rep news assessments and Pro-Dem news assessments. Using R 2 , variation in news assessments explains 13 percent of the variation in beliefs. By comparison, all of the non-political demographics collected in this experiment -age, gender, race, education, logged income, whether one is religious, and whether one is from a state that voted for Trump or Clinton in 2016 -explain 7 percent of the variance in beliefs.
32 This suggests that my measure of motivated reasoning performs at least as well as oft-discussed demographics at predicting beliefs. Second, motivated reasoning can help explain a particular form of overprecision due to miscalibrated confidence intervals. In particular, we posit that motivated reasoners formed directionally-biased belief distributions, thereby leading them to overestimate the probability that the answers are within their confidence interval. Such a story would imply that overprecision is stronger for politicized than for neutral topics, and stronger for partisans than for moderates: though this may simply be due to the non-representativeness, conditional on party, of the MTurk sample. For instance, only 76 percent of Republicans in this sample approved of President Trump's performance; in a Gallup poll conducted contemporaneously (from June 25-July 1), 87 percent of Republicans approved of his performance (Gallup 2018). 32 These are unadjusted values. Adjusted R 2 is 13 percent for assessments and 6 percent for demographics. • On politicized and performance questions, subjects' 50-percent confidence intervals contain the correct answer less than 50 percent of the time. • On politicized questions, the likelihood that subjects' confidence intervals contain the correct answer decreases in their partisanship. In support of Hypothesis 4, subjects are overprecise in their beliefs about questions that evoke motivated beliefs. On politicized topics, subjects' confidence intervals contain the correct answer 46.6 percent of the time (s.e. 0.6 percent); this is statistically significantly less than 50 percent (p < 0.001). Overprecision on these topics is primarily driven by partisans, whose intervals contain the correct answer 44.2 percent of the time (s.e. 0.9 percent). Moderates' intervals contain the correct answer 48.8 percent of the time (s.e. 0.8 percent). Partisans' overprecision is statistically significantly larger than moderates' (p < 0.001). On the performance question, subjects' confidence intervals contain the correct answer 42.0 percent of the time (s.e. 1.6 percent), which is statistically significantly less than 50 percent (p < 0.001). Overprecision is stronger for motivated questions than neutral questions, suggesting that these results are not simply driven by a bias towards overly narrow confidence intervals. On the "Random Number" question, which asks subjects to guess what a random number drawn uniformly from 0 to 100 will be, confidence intervals contain the correct answer 54.6 percent of the time, which indicates mild underprecision. 33Finally, overprecision is correlated with over-trusting error-reinforcing Fake News, consistent with the notion that current beliefs are reflective of motivated beliefs. Subjects who are overprecise on a question assess Fake News to be 3.3 pp more likely to be truthful (s.e. 0.8 pp; p < 0.001) compared to underprecise subjects, and subjects who are overprecise on a question assess True News to be 2.4 pp less likely to be truthful (s.e. 0.7 pp; p = 0.001) compared to underprecise subjects. The experimental results strongly favor the hypothesis of motivated reasoning with politically-motivated beliefs over the null hypothesis of Bayesian updating. Motivated reasoning also fits the data better than unmotivated patterns of over-or underinference. Subjects significantly over-trust Pro-Party news and Fake News in an environment with uninformative signals, real monetary stakes, and little room for self-deception. These results show that political motives are necessary to consider in addition to prior beliefs when predicting how people respond to new information. In fact, motivated reasoning can partly explain one why people update further in the direction of their prior than a Bayesian would. When priors reflect motivated beliefs, a method that detects prior-confirming biases in belief updating may be detecting motivated reasoning instead (e.g. Eil and Rao 2011). The results in Section 4.3 relate to the effect of motivated reasoning on political polarization. Not only do subjects polarize in beliefs about the veracity of news, they polarize in their beliefs about the questions themselves, despite receiving uninformative signals. Gentzkow and Shapiro (2011) find only modest differences in the media that liberals and conservatives consume, and motivated reasoning can help explain why people polarize even if they consume similar media outlets. Results also suggest that the beliefs people are motivated by are not necessarily optimistic ones, but are rather ones that support their political identity.34 Furthermore, these results indicate that beliefs people find attractive are even further apart than their current beliefs. One reason that people do not already hold their most-preferred beliefs is that motivated reasoners are still affected by information; the amount of distortion in updating is constrained by actual informational content. Motivated reasoners who receive precise signals would in fact become less polarized. This paper has shown that distortion of new information in favorable directionsmotivated reasoning -plays a substantial role in people's assessment of the veracity of news and helps explain why people form inaccurate and polarized beliefs about the world around them. It developed a novel experimental paradigm that is able to identify the channel of motivated reasoning from Bayesian updating and other channels across a variety of settings. Results demonstrated the role that politicallymotivated reasoning plays in how people form beliefs about applied economic issues like income mobility, crime, and immigration. They also showed how this bias can lead to further belief polarization, overprecision, and an excess trust in Fake News. There are several avenues for future work with this design. For instance, the design can be used by applied researchers who are interested in detecting motivated reasoning. Researchers, working in many different contexts, can plug in any factual question with a real-valued answer to determine how people motivatedly reason about that question. Motives can be compared across varied contexts and populations. The notion of a susceptibility parameter also suggests a lever for debiasing people. This design enables us to identify and estimate the magnitude of motivated reasoning; interventions whose objective is debiasing can then use this estimate to test the efficacy of the treatment. One approach is to estimate susceptibility for a treatment group and a control group. Having an approach to reducing motivated reasoning would be a valuable way for researchers to combat polarization and biased beliefs about important issues, especially in a highly politicized society.
Taber, Charles and Milton Lodge (2006). "Motivated Skepticism in the Evaluation of Political Beliefs". In: American Journal of Political Science. Tappin, Ben, Gordon Pennycook, and David Rand (2020a). "Bayesian or biased? Analytic thinking and political belief updating". In: Cognition. Tappin, Ben, Gordon Pennycook, and David Rand (2020b). "Thinking clearly about causal inferences of politically motivated reasoning: Why paradigmatic study designs often undermine causal inference". In: Current Opinion in Behavioral Science. Tetlock, Philip (1983). "Accountability and the perseverance of first impressions". In:

Some people believe that the Obama administration was too soft on crime and that violent crime increased during his presidency, while others believe that President Obama's pushes towards criminal justice reform and reducing incarceration did not increase violent crime. This question asks how murder and manslaughter rates changed during the Obama administration. In 2008 (before Obama became president), the murder and manslaughter rate was 54 per million Americans. In 2016 (at the end of Obama's presidency), what was the per-million murder and manslaughter rate? Correct answer: 53. In 2017, Donald Trump signed into law the largest tax reform bill since Ronald Reagan's 1981 and 1986 bills. Some people believe that Reagan's reforms accelerated economic growth and allowed lower-income Americans to reap the benefits of lower taxes, while other people believe that this decreased the government's spending to help lower-income Americans get ahead. This question asks whether children who grew up in low-income families during Reagan's tenure were able to benefit from his tax reforms. Of Americans who were born in the lowest-income (bottom 20%) families from [1980][1981][1982][1983][1984][1985], what percent rose out of the lowest-income group as adults? (Please guess between 0 and 100.) Correct answer: 64.9. Source linked on results page: http: // bit. ly/ us-upward-mobility (page 47)

In the United States, white Americans have higher salaries than black Americans on average. Some people attribute these differences in income to differences in education, training, and culture, while others attribute them more to racial discrimination. In a study, researchers sent fictitious resumes to respond to thousands of help-wanted ads in newspapers. The resumes sent had identical skills and education, but the researchers gave half of the (fake) applicants stereotypically White names such as Emily Walsh and Greg Baker, and gave the other half of the applicants stereotypically Black names such as Lakisha Washington and Jamal Jones. 9.65 percent of the applicants with White-sounding names received a call back. What percent of the applicants with Black-sounding names received a call back? (Please guess between 0 and 100.) Correct answer: 6.45. Source linked on results page: http: // bit. ly/ labor-market-discrimination

In the United States, men are more likely to enter into mathematics and math-related fields. Some people attribute this to gender differences in interest in or ability in math, while others attribute it to other factors like gender discrimination. This question asks whether high school boys and girls differ substantially in how well they do in math classes. A major testing service analyzed data on high school seniors and compared the average GPA for male and female students in various subjects. Male students averaged a 3.04 GPA (out of 4.00) in math classes. What GPA did female students average in math classes? (Please guess between 0.00 and 4.00.) Correct answer: 3.15. Source linked on results page: http: // bit. ly/ gender-hs-gpa

Some people believe that the U.S. has a responsibility to accept refugees into the country, while others believe that an open-doors refugee policy will be taken advantage of by criminals and put Americans at risk. Some people believe that there is a scientific consensus that human activity is causing global warming and that we should have stricter environmental regulations, while others believe that scientists are not in agreement about the existence or cause of global warming and think that stricter environmental regulations will sacrifice jobs without much environmental gain. This question asks about whether most scientists think that global warming is caused by humans. A major nonpartisan polling company surveyed thousands of scientists about the existence and cause of global warming. What percent of these scientists believed that "Climate change is mostly due to human activity"? (Please guess between 0 and 100.) Correct answer: 87. Source linked on results page: http: // bit. ly/ scientists-climate-change

The United States has a homicide rate that is much higher than other wealthy countries. Some people attribute this to the prevalence of guns and favor stricter gun laws, while others believe that stricter gun laws will limit Americans' Second Amendment rights without reducing homicides very much. After a mass shooting in 1996, Australia passed a massive gun control law called the National Firearms Agreement (NFA). The law illegalized, bought back, and destroyed almost one million firearms by 1997, mandated that all non-destroyed firearms be registered, and required a lengthy waiting period for firearm sales. Democrats and Republicans have each pointed to the NFA as evidence for/against stricter gun laws. This question asks about the effect of the NFA on the homicide rate in Australia. In the five years before the NFA (1991)(1992)(1993)(1994)(1995)(1996), there were 319.8 homicides per year in Australia. In the five years after the NFA (1998)(1999)(2000)(2001)(2002)(2003), how many homicides were there per year in Australia? Correct answer: 318.6. Sources linked on results page: http: // bit. ly/ australia-homicide-rate (Suicides declined substantially, however. For details: http: // bit. ly/ impact-australia-gun-laws .) Some people believe that the media is unfairly biased towards Democrats, while some believe it is balanced, and others believe it is biased towards Republicans. This question asks whether journalists are more likely to be Democrats than Republi-

This question asks whether you think Democrats or Republicans did better on this study about political and U.S. knowledge. I've compared the average points scored by Democrats and Republicans among 100 participants (not including yourself). The Democrats scored 72.44 points on average. How many points do you think the Republicans scored on average? (Please guess between 0 and 100)

Correct answer: 70.83.
How well do you think you performed on this study about political and U.S. knowledge? I've compared the average points you scored for all questions (prior to this one) to that of 100 other participants. How many of the 100 do you think you scored higher than? (Please guess between 0 and 100.) Agents may have different beliefs because these beliefs reflect past instances of motivated reasoning. In this subsection, we consider motives that depend on the state θ q and an econometrician who observes motivated-reasoning agents' updating process, inferring whether their motive function is increasing or decreasing in θ q . When motives are unobservable, an experimenter can learn about agents' motives by looking at their initial beliefs µ iq . Conceptually, an agent's error in beliefs can be partly explained by motivated reasoning, and therefore the direction of the error predicts the direction of the motive function. Two agents who motivatedly reason in different directions, and each who receive a signal drawn from the same distribution, will hold different median beliefs: A motivated reasoner with an increasing motive function will be more likely to hold a belief that µ iq > θ q , and a motivated reasoner with a decreasing motive function will be more likely to hold a belief that µ iq < θ q . This implies that an agent who believes µ iq > θ q is more likely to have an increasing motive function than is an agent who believes µ iq < θ q . When the two agents then make news assessments using the structure above, agents will trust news that reinforces the error in their beliefs more than news that mitigates the error. This occurs even though signals are designed exactly so that their interpretation is distinct from µ iq . More formally, there is a state θ q ∈ R. Consider a Bayesian (she) and a motivated reasoner (he) with the same prior θ q ∼ F θq and who receive a public signal z q ∼ F zq (•). We now assume that motivated reasoners have heterogeneous motive functions m iq (θ q ) that are strictly monotonic in θ q ; that is, m iq (θ q ) is drawn from a distribution F mq (m q (•)) that includes strictly increasing and strictly decreasing functions of θ q . The econometrician observes the distributions F zq (•) and F mq (m q (•)), as well as the states θ q and reported medians µ iq , but not the realizations z q or m iq (θ q ). We also assume that z q leads the Bayesian's posterior median µ Bq to take values close to θ q with positive probability, but that there is no point mass at exactly θ q . That is, for all δ > 0, there exists some δ > 0 such that P(|µ Bq -θ q | < δ) > δ , while P Bq (µ Bq = θ q ) = 0. Without loss of generality, consider what the econometrician infers about a motivated reasoner who has a strictly increasing m iq (θ q ). Since the log-likelihood of the motive is strictly increasing, his posterior distribution first-order stochastically dominates the Bayesian's posterior distribution. In addition, for every such motive function, there exists a δ > 0 such that for all signals leading to the Bayesian having a posterior median µ Bq ∈ (θ q -δ, θ q ), the motivated reasoner has posterior median µ iq > θ q . Since there is a probability of at least δ > 0 of such a signal, this high-θ qmotivated reasoner is strictly more likely than the Bayesian to state a median belief that is greater than θ q . By the same argument, a low-θ q motivated reasoner is strictly less likely than the Bayesian to state a median that is less than θ q . If some people have monotonically-increasing motives and others have monotonicallydecreasing motives, then the econometrician will believe that:

Recall that message G iq says that θ q > µ iq and that message L iq says θ q < µ iq . Then, for the econometrician,

when motives are heterogeneous. Since G iq and L iq are equally likely by construction, this means that agents trust the source of error-reinforcing messages more than the source of error-mitigating messages on questions when motivated reasoning plays a role. Using this signal structure, error-mitigating messages come from True News and error-reinforcing messages come from Fake News. Therefore, agents give higher assessments to Fake News than True News, with and without controls for observable party preference: Fact (Motivated reasoning leads to over-trusting Fake News, under-trusting True News) Suppose that agents motivatedly reason with a strictly monotonic motive. Then:

• a(Fake News iq ) > a(True News iq ). • a(Fake News iq ; Pro-Party news iq ) ≥ a(True News iq ; Pro-Party news iq ). • a(Fake News iq ; Anti-Party news iq ) ≥ a(True News iq ; Anti-Party news iq ). Suppose also that the sign of the slope of the motive function is heterogeneous within party. That is, the probability of an agent having ∂m iq (θq) ∂θq > 0 is strictly between 0 and 1, conditional on the agent's party. Then:

• a(Fake News iq ; Pro-Party news iq ) > a(True News iq ; Pro-Party news iq ). • a(Fake News iq ; Anti-Party news iq ) > a(True News iq ; Anti-Party news iq ). The stark result that motivated reasoners will trust Fake News more than True News is particular to the uninformativeness of the messages. However, the prediction that agents will trust Fake News more than a Bayesian will is quite general, only relying on unobservable inputs into current beliefs. This prediction only holds for motivated states, psychologically differentiating this theory from unmotivated explanations of over-trusting Fake News (such as confirmation bias).
Practically, it suggests that excessive trust in disinformation is more prominent when people hold motivated beliefs. Under further functional form assumptions, the model of motivated reasoning also predicts that agents will overestimate the probability that the answers to the questions are within a subjective confidence interval: overprecision. Unlike other determinants of overprecision, this will be because of miscalibrated beliefs due to motivated reasoning, and we do not assume a general bias towards overly-narrow confidence intervals. Unlike the previous subsection, we now assume that Nature draws θ q from N (µ q0 , 1/τ 2 q0 ) independently across q, and that Nature gives agents a noisy public signal z q = θ q + qz , where qz ∼ N (0, 1/τ 2 qz ). Slightly abusing notation, we assume also that motivated reasoners have motives linear in θ q : m iq (θ q ) = m iq • θ q . 35 In the political context, |m iq | can be thought of as political partisanship. Finally, we additionally assume that the prior probability of True News is p and is constant for all agents. After observing z q (but before playing the game in Section 2.1), a Bayesian forms the posterior: 35 We also keep the assumption of constant ϕ but note that there are many reasons to believe that ϕ ought to depend on z q . With normally-distributed signals, one way that ϕ could depend on z q is through its precision. For instance, ϕ(z q , τ qz ) = min{ϕ c τ qz , φ}. Using this form, the susceptibility of two weak signals is equal to the sum of their precisions, but there is a maximum level of susceptibility. and a motivated reasoner forms the posterior:

Notably, the two agents have the same posterior variance, but the motivated reasoner's distribution is miscalibrated. Consider their (1 -Q)/2-quantile and (1 + Q)/2-quantile beliefs, and call this the Q-confidence interval. Then:

Suppose that a motivated reasoner has normally-distributed priors and receives a signal normally distributed with mean equal to θ q , as above. When ϕ > 0, the probability that his Q-confidence interval contains θ q is equal to Q for m iq = 0 and strictly decreases in |m iq |. Since Bayesian updating is equivalent to motivated reasoning with m iq = 0, Bayesians will appear to be appropriately precise and motivated reasoners will appear overprecise due to their miscalibrated confidence intervals. They will be both be appropriately precise on neutral topics, where m iq = 0. Note that the direction of the bias relies both on linear motives and the normal-normal functional form. 36In the context of politics, I posit that partisanship is a proxy for |m iq | on politicized questions. On such questions, the probability that agents' Q-confidence interval contains θ q will then be decreasing in their partisanship. As seen in the first column of Table 9, results suggest that subjects are more likely to update in the direction of the Pro-Party message compared to the Anti-Party message (p = 0.055). 39 The third column shows that, as in Section 4.4, this difference vanishes once the news veracity assessment measure is controlled for. The underperformance result (not discussed in the main text) is strongly replicated. Subjects score 66.3 points (s.e. 0.4 points) on politicized news assessments and 65.5 points (s.e. 1.6 points) on performance news assessments on average. Both of these are statistically significantly lower than 75 points, the score that subjects would receive if they had answered "5/10 chance the message came from True News" (p < 0.001). The result that subjects' confidence intervals are overprecise is strongly replicated. On politicized topics, subjects' 50 percent confidence intervals contain the correct answer 44.1 percent of the time (s.e. 0.8 percent); this is statistically significantly different from 50 percent (p < 0.001). As seen in Table 10, the result that this measure of overprecision is increasing in partisanship is suggestive (p = 0.066). The two polarization results are replicated. On politicized topics, Table 9 shows that subjects are statistically significantly more likely to follow Polarizing news than anti-Polarizing news (p = 0.031). 40 Subjects also state initial medians that are more likely to be in the Pro-Party direction (p < 0.001). I did not pre-register replication tests for performance-driven motivated reasoning (or anything involving neutral topics) given the limited sample size. Results, however, are broadly similar to those in the main experiment. Subjects assess Pro-Performance news to be 8.0 percentage points higher than Anti-Performance news (s.e. 2.6 percentage points; p = 0.003). Demographic heterogeneity, robustness exercises, and minor results were also not tested. Further work can test whether these results also replicate with a larger sample. 39 As with the main effect, the coefficient is smaller in the replication, due in large part to the new questions. On the questions with the exact same wording as the original study, the treatment effect is 5.7 percentage points (s.e. 2.6 percentage points). On other politicized questions, the treatment effect is 2.0 percentage points (s.e. 2.6 percentage points). 40 As in Section 4.4, this difference vanishes once the news assessment measure is controlled for. Notes: OLS, errors clustered at subject level. Only Pro-Party / Anti-Party news observations. Polarizing News: tells subjects that, compared to their initial guess, the answer is in the opposite direction from the population mean. Dependent variable is 1 if subjects change their guess upwards when the message says "Greater Than" or downwards when the message says "Less Than," -1 if they change it in the opposite direction, and 0 if they do not change it. This section presents results on heterogeneity in magnitude of motivated reasoning and additional robustness checks for the main results in Table 2. Results are similar for each randomization arm, if I include subjects who fail comprehension checks, and if the dependent variable is the logit probability of news veracity assessments.
Figure 8 plots the coefficients from the regression of news assessments on the interaction of whether the news was "good" or "bad" and partisanship, as well as on binarized observable demographics. We see that the effect of non-political demographics are small, and most are statistically insignificantly different from zero. Notes: OLS, errors clustered at subject level. Neutral News indicates that Pro-Party / Anti-Party news assessments are compared to assessments on Neutral topics. These classifications are in Table 1. Controls: race, gender, log(income), years of education, religion, and whether state voted for Trump or Clinton in 2016. Partisanship is abs(Republican Party rating -Democratic Party rating). Notes: OLS, errors clustered at subject level. Only subjects from the Second-Guess group. Notes:

Notes:

Notes:Notes:Notes:

Notes:Notes:

Notes:

There is ample evidence consistent with these channels (e.g. Taber and Lodge

2006; Kahan et al. 2012; Nyhan and Reifler 2010; Nyhan and Reifler 2013; Nyhan, Reifler, and Ubel 2013). Mayraz (2019) uses a similar formulation in which motivated beliefs affect priors instead of belief updating. We assume that there is no atom at µ iq and that P(µ iq = θ q ) = 0. That is, the agent believes that the answer has probability zero of being exactly equal to µ, and the probability is indeed zero. Neutral questions ask about the latitude and longitude of the center of the U.S. and about a random number drawn from 0-100. The comprehension check asks what the current year is. This is not the case for the comprehension check question, where the message says "The answer is equal / not equal to your previous guess of [My Guess]." I find that subjects have a positive valuation of these messages, but that they do not differentially value messages on politicized and neutral topics; results suggest that subjects are aware that they treat signals as informative, but not that they expect to distort their updating process. The low frequency of correct guesses indicates that subjects were generally not looking up answers. It also suggests that the model's assumption of an atomless belief distribution is reasonable. The replication was identical in structure, but added several new political questions and did not include neutral questions, so it is not easy to pool results. All main hypotheses on political questions were preregistered, as were results on polarization and overprecision. If anything, assessments are higher for True News than Fake News on neutral topics. Reflecting on the question may lead subjects to adjust towards the truth in the absence of motivated beliefs. 99.4 percent of non-neutral subjects receive at least one piece of Pro-Party and Anti-Party news. Three subjects randomly never receive Pro-Party news and two never receive Anti-Party news. The Online Appendix shows that results are qualitatively the same if we replace a iqr with logit(a iqr ). The mean time spent on the assessment page with Pro-Party news is 14.6 seconds (s.e. 0.3 seconds), and the mean time spent on the assessment page with Anti-Party news is 14.8 seconds (s.e. 0.3 seconds). There is suggestive evidence that Pro-Dem subjects motivatedly reason than Pro-Rep subjects,

On the other neutral questions, subjects also exhibit moderate underprecision. In fact, Thaler (2020)  shows that absent politics, optimism about the world plays a negligible role in motivated reasoning. For instance, suppose that the state space contains two values and a Bayesian infers from a signal that either one value has a (1 + Q)/2 likelihood of occurring or the other value has a (1 + Q)/2 likelihood of occurring. Then, her confidence interval would contain one point, and a motivated reasoner may have a confidence interval that contains both points.