Refereed Original Paper
Literacy is necessary to understand Fact-Checking: An
empirical research using survey experiments
Keywords:
Fact-checking, Media literacy, Information literacy, Backfire effect, Social media

Tsukasa TANIHARA, Kanda University of International Studies
Shinichi YAMAGUCHI, International University of Japan

Abstract
Regarding the impact of fact-checking, extensive research has been conducted on thecorrelation between fact-checking and individuals’ political beliefs, but this issue is difficult toaddress by policy. This study investigates the relationship between the effectiveness of factchecking and literacy, as well as the relationship between the effectiveness of fact-checking andthe types of media used to disseminate this information. These variables can be addressed
through policy measures. We conducted the survey via the internet. Participants were tasked
with making true or false judgments about real instances of misinformation before and after factchecking. The results highlighted the significance of information literacy in achieving accurate
perceptions through fact-checking. Secondly, in the case of COVID-19-related misinformation,fact-checking proved more effective on government websites than on social media. Thirdly, many
individuals incorrectly identified misinformation as true even after fact-checking. These findings
underscore the risk of indiscriminately disseminating fact-check results on social media, as doing
so could potentially have the opposite effect if the recipients lack the requisite literacy.

Received May 20 2022, Accepted Feb 8 2023

33

Literacy is necessary to understand Fact-Checking: An empirical research using survey experiments

TANIHARA et al

72% of respondents reported exposure to COVID-19-

1. Introduction

related misinformation (Ministry of Internal Affairs
This study aims to foster the practice of fact-

and Communications, 2020). A separate survey

checking. Specifically, we identify the relationshipcarried out in Japan in 2020 showed that 28.2% of

between fact-checking and literacy of information,

participants

as well as effective media for broadcasting fact-misinformation, while 45.2% had been exposed tochecking. These matters can be addressed

COVID-19-related misinformation (Yamaguchi etthrough policy measures, and their implicationsal., 2020). As indicated, misinformation presents afor society are significant. This study is notpressing concern in both political and COVID-19

intended to recommend that the government

contexts.

reportedencountering

political

actively intervene in fact-checking. The policy

Since the launch of the Fact Check Initiative

responses proposed in this study are primarily

Japan (FIJ) in 2017, numerous media entities andintended

media

experts have cooperated to carry out fact-organizations, platform companies, and non-checking. However, these efforts have not fully

profit organizations.

permeated society. In Japan, 71% of survey

for

implementationbyFact-checking refers to the process of examining

respondents reported never having heard thethe accuracy of information, news, or statements

term “fact-checking” (Yamaguchi et al., 2020).

circulating in society, documenting that process inGiven this situation, it is crucial to contemplatearticles, and sharing accurate information withsuitable policy responses to facilitate smootherpeople. This is an essential initiative in today’s

dissemination of fact-checking results withinsociety as it is inundated with misinformation. The

society.

issue of misinformation gained prominence during

2. Research question

the 2016 U.S. presidential election. Misinformation
supporting Trump was shared a total of 30 million

2.1 Literature review

times, while misinformation supporting Clinton
was shared a total of 8 million times (Allcott andIn recent years, several empirical studies haveGentzkow, 2017). Amid the COVID-19 pandemic,

been conducted on fact-checking and debunking

the World Health Organization (WHO) declared

information (for meta-analyses, see Lewandowsky

misinformation

et al., 2012; Walter et al., 2020; Walter andan“infodemic,”aproblemMurphy).

requiring attention comparable to the infectious

The issues presented are as follows. First, it isdiseases themselves.
Japan grapples with a plethora of misinformation.

related to individuals’ beliefs, particularly political

For instance, during the 2020 U.S. presidential

beliefs. That is, the effect of corrective informationelection, misinformation suggesting that Mr. Biden

depends on whether its content is consistent with

was rigging the election pervaded Japan. Moreover,an individual’s political beliefs (Nyhan andcountering misinformation has evolved into an

Reifler, 2012; Swire et al., 2017; Wood and Porter,

increasingly critical issue amid the pandemic.

2018). Naturally, pro-attitudinal information is

According to a survey conducted by the Ministry of

more effective (Walter et al., 2020). This is called

Internal Affairs and Communications, approximately

motivated reasoning (Nyhan and Reifler, 2012). It
34

Journal of Socio-Informatics Vol. 16 No. 1 Sep. 2023

has also been shown that fact-checking is

Vraga et al. (2020) and Tanihara et al. (2022)

sometimes selectively ingested and spread based

conducted two of the few existing studies on thison an individual’s belief (Shin and Thorson,

subject. Vraga et al. (2020) performed an2017).

experiment where subjects were exposed to tweets

Second, the backfire effect was noted in relation

containing misinformation and their corrections,to this point. The backfire effect occurs when

following tweets that invoked news literacy. This

individuals

results

study, however, did not establish that invoking

conflicting with their beliefs, which consequently

news literacy significantly influences information

strengthens

encountertheirfact-checking

original

correction. The authors posited that tweets

Republican

invoking news literacy may have been lost in the

supporters of the U.S. invasion of Iraq in 2003,

timeline noise. Tanihara et al. (2022), employing

even after correcting that the information about

the categorization by Jones-Jang et al. (2021),

the existence of Weapons of Mass Destruction

divided literacy into the aforementioned four

(WMD) in Iraq was false, still strengthened theircategories and investigated their influence on themisbelief in the misinformation about theeffect of fact-checking. The results demonstrated

existence of WMD in Iraq (Nyhan and Reifler,that individuals who reconsider their views when2010). The above points are based on a study offact-checking results are presented via mass media

how an individual’s political beliefs (liberal ortend to have higher news literacy. In contrast, those

conservative) relate to the effects of corrective

who change their minds when fact-checking

information. Despite great academic interest,

results are presented through social media exhibit

there are some challenges concerning policy

low information literacy. A significant issue in this

implications. This is because policymakers

study is that media and news literacy were self-

generally cannot manipulate individual beliefs.

reported and not adequately measured. It is also

misinformation.

For

faith

in

example,the

Therefore, the third and fourth points are theperplexing that individuals with low information

focus of this study. The third point relates to literacy,

literacy tend to accept fact-checking results fromas it relates to information. If we can clarify the typesocial media more readily than those with highof literacy involved in accepting fact-checking, it will

information literacy. Given that the survey design

be possible to implement policy responses.

did not explicitly identify the corrected information

Regarding the relationship between misinformation

as fact-checking, Tanihara et al. (2022) suggested

identification and literacy, Jones-Jang et al. (2021)

that individuals who alter their views based on

astutely categorize literacy into four domains—corrected information from social media arenews literacy, media literacy, information literacy,

susceptible to the information and tend to change

and digital literacy—and investigate the effects of

their minds without thorough examination.

each. The results indicate that only information

Fourthly, the medium’s impact differs: in line

literacy significantly contributes to the identification

with McLuhan’s (1964) assertion “the medium isof misinformation.

the message,” reactions may vary based on theOn the other hand, few studies have considered

mediumdeliveringcorrective

information.

literacy as a variable in examining the effect of fact-

Identifying the most effective media for fact-

checking, and the results have been inconsistent.

checking could enable more potent policy
35

Literacy is necessary to understand Fact-Checking: An empirical research using survey experiments

TANIHARA et al

responses, focusing on highly effective platforms.

As both RQ1 and RQ2 do not have consistent

Tanihara et al. (2022) offered insights here but

effects in previous studies, we do not set specificonly differentiated between mass and social

hypotheses. We examine the effects of each

media, without inferential statistical analysis ofvariable in an exploratory manner.

the two.

3. Methodology
2.2 Research gaps
Participants

The research gap in related studies, withrespect to policy implications, can be summarizedThis study utilized data from an internet survey

as follows. First, many studies have focused on

conducted in Japan in 2022 by the Center forvariables that are difficult to address in policy

Global Communications at the Internationalterms, such as individual political beliefs. Second,

University of Japan. The survey period spanned

the few studies examining relationship withfrom February 18 to February 23, 2022. The

literacy have produced inconsistent results.

survey was disseminated to 8,394 respondents via

Third, the identification of specific media types

My Voice.com Inc. and its partners, garnering

where fact-checking is effective is lacking.

5,987 responses. After eliminating responses

Therefore, this study contributes to related

from samples that answered cursorily as indicatedresearch and policy by adopting the following

by trap questions, the final sample size was set atapproach. First, we designed a survey experiment5,569. This survey, however, was designed suchthat transforms literacy items into a test format

that about 90% of the respondents had beenand specifies that the presented corrective

exposed to the misinformation presented in theinformation has undergone fact-checking. Second,preliminary survey. Consequently, the analysis

we categorized the media into four groups (online

was weighted according to the percentage ofnews, social media, newspapers, and government

respondents who had encountered misinformation

websites) and randomly divided participants into

in the pre-survey. In the preliminary survey, 40.9%

these groups for the survey experiment. Asof the respondents encountered one or morediscussed below, by incorporating an interaction

pieces of misinformation. Conversely, in the main

term for each medium into the regression model,

survey, they accounted for 90.1%. Therefore,

we can identify individual characteristics thatweighting was employed to adjust for this

respond to fact-checking and the differential

discrepancy. The sample sizes are presented in

effects of each medium. The following research

Table 1.

questions (RQs) were set for this study based on

Table 1 Sample size

the above:
RQ1: How does individual literacy influence theeffectiveness of fact-checking?
RQ2: Are there differences among media
concerning the effectiveness of fact-checking?

36

Journal of Socio-Informatics Vol. 16 No. 1 Sep. 2023

Research Design

abolition of plastic bag fees, stating, “There is no

The misinformation utilized in the survey
experimentreflects

actualsuch fact.” In light of this information, how would

misinformation

you reconsider the following news? “The current

circulated in society. We specifically employed thecabinet decided to abolish the plastic bag fee

following two types of misinformation, both ofsystem.”which were determined to be “False” by factchecking conducted by FIJ’s partner organizations:

For the remaining three groups, the questions
were posed by replacing “was published in an
online news article” with “was posted on social

1.The current cabinet decided to abolish themedia,” “was published in a newspaper article,”plastic bag fee system (related to politics).
2.The French government began vaccinatingand “was posted on a government website,”children against the coronavirus without parental

respectively. The options were “1 = It is correctconsent starting in September 2021 (related to

information,” “2 = It is misinformation /COVID-19).

unsubstantiated information,” and “3 = Do notknow / cannot say either.” Those who chose 2

Participants who were already aware of the above

were accurate, indicating that they had arrived at

two news items before the survey were excluded

the correct perception of the information. In thefrom the analysis. Participants encountering theprocess described above, participants werenews items for the first time were asked whether

queried about their opinion on the two pieces ofthey perceived the news to be accurate. The choices

misinformation before and after fact-checking.

were: “1 = It is correct information,” “2 = It is

After the survey’s completion, respondents were

misinformation / unsubstantiated information,” andnotified by email that all listed news were false.

“3 = Do not know / cannot say either.” Subsequently,

Cross tabulation

the respondents were presented with corrective
information regarding the news and asked again

The results of the survey experiment areabout their perception of it. Specifically, we createddepicted in Tables 2 and 3. Table 2 indicates the

scenarios in which corrective information was

respondents’ perceptions of the news without theposted on online news, social media, newspapers,

presence of fact-checking. Concerning theand government websites, and assigned the sample

political

to each of the four randomly divided groups. For

respondents selected “It is misinformation /instance, the group for whom the correction

unsubstantiated information,” suggesting that

information was posted on online news responded

over half of the respondents identified theto the following questions:misinformation. In regard to the COVID-19-

news,

approximately

62%

of

the

related news, the distribution of responses prior
The following information was published in an

to the introduction of fact-checking revealed thatonline news article. The Recycling Promotion

over half of the respondents, around 59%,

Office of the Ministry of the Environment, theanswered “Do not know / cannot say either.”department in charge, categorically denied that theThis implies that forming a judgement on thiscurrent cabinet had made a decision regarding thewas more challenging compared to political37

Literacy is necessary to understand Fact-Checking: An empirical research using survey experiments

TANIHARA et al

Table 2 Cross-tabulation (before fact-checking)Table 3 Cross-tabulation (after fact-checking)news.

both news categories, even when confronted with

Table 3 depicts the distribution of judgments

fact-check results, some respondents persisted inafter exposure to fact-checking through various

their belief in the original information. While themedia. As for the politics-related news, While thecorrected information seems to have assisted inpercentage of respondents answering “Do notdiscerning the veracity of the information, it did

know / cannot say either” for political news

not necessarily result in correct comprehension.

decreased,

respondents

As it is crucial to elucidate this situation, further

maintaining incorrect beliefs increased. The

analysis will be conducted in subsequent sections.

percentage of respondents attaining correct

In terms of the medium, the percentage ofcomprehension was slightly reduced post-correct responses for political news increased incorrection. In terms of COVID-19-related news,

the sequence of online news < social media <the percentage of those responding “Do notnewspapers < government websites. For COVID-

know / cannot say either” decreased significantly.

19-related news, the percentage of correct

While the number of respondents stating, “It is

responses escalated in the order social media <misinformation / unsubstantiated information”

online news < newspapers < government(indicating correct comprehension) increased,websites. However, in both instances, thethe percentage who answered, “It is correct

difference is minor, and a thorough analysis in

information” also rose.

the upcoming section is required to determine its

theproportionofThis presents the surprising outcome that instatistical significance.
38

Journal of Socio-Informatics Vol. 16 No. 1 Sep. 2023

Regression Analysis

The foregoing results reveal an unexpected
increase in the percentage of individuals who

Building on the results from the previous

incorrectly identified misinformation as “correct”

section, we will conduct a regression analysis toin both political and COVID-19-related news, evendiscern the characteristics of those forming

after undergoing fact-checking. To delve further

correct or incorrect perceptions based on fact-

into this issue, Table 4 presents a detailed accountchecking. The method of variable creation is asof the pre-fact-checking decisions made by thosefollows.

who

misidentified

misinformation

post-fact-

checking. For instance, approximately 46.3% ofIndependent Variables

those who initially labelled the COVID-19-related

Media literacy and information literacy

misinformation as “correct information” continued

To address RQ1, we established literacy as an

to do so even after exposure to fact-checking. This

independent variable, incorporating two forms ofphenomenon may be explained by the previously

literacy: media and information. For media literacy,

mentioned backfire effect, wherein respondents

we referenced Kodera (2017), who structured

initially misled by misinformation remain misled,both domestic and international measurements offact-checked information, thereby sustaining their

media literacy. The questions, presented in a test-

original misconception. However, the backfire

like format, gauged respondents’ understanding

effect fails to explain the cases of those respondents

of six properties of media: the constructive nature

who initially identified the news as “misinformation/of media messages, the media’s capacity to shapeunsubstantiated information” or “do not know/“social reality,” the commercial aspects of media,

cannot say either”, but later reversed their decision

the ideological and value-based transmission of

to

In

these

cases,

media, media style and language, and the non-renderedthecorrect

uniform interpretability of the recipient. Media

judgment, but subsequently arrived at an incorrect

literacy measurements are as follows. Respondents

understanding after encountering fact-checking.

were asked to respond to each of the following

We tentatively term this the “adverse effect of fact-

questions using a four-point scale, from “1 =checking”. To illuminate the characteristics ofstrongly agree” to “4 = strongly disagree”. The

individuals who develop incorrect perceptions

questions were presented in random order.

“correct

respondents

information”.
initially

after fact-checking, we proceeded with a regression[1] Online news is neutral and objective.

analysis.

[2] The “average person” depicted in the news
represents an average Japanese individual.

Table 4 Cross-tabulation (before and after fact-checking)39

Literacy is necessary to understand Fact-Checking: An empirical research using survey experiments

[3] The range of opinions online closely reflects

TANIHARA et al[2] Restaurant X opened in 2005, [3] Don’t visit

societal reality.

restaurant X because it isn’t good, [4] Restaurant[4] Television programs are produced with

X should discontinue 24-hour operations for theviewer responses in mind (reverse-coded).

health of its employees[5] TV and internet content can be influenced by
the preferences of sponsors (reverse-coded).

(3) Which of the following statements about[6] The media provides viewers with value

Shohei Otani does not include the author’s

judgments of what is “good” and “bad”

opinion? Select one.

(reverse-coded).

[1] Shohei Otani is an exceptional baseball[7] The same event is reported identically inplayer, [2] Shohei Otani improved by moving tomass media (newspapers, TV, etc.) and online

America, [3] Shohei Otani revolutionized thenews.

world of baseball, [4] Shohei Otani declined the[8] Different images used in the same TV news

National Medal of Honor.

broadcast can generate different impressions(reverse-coded).

(4) Here is news about the Amazon rainforest:“In 2019, the total area of the Amazon rainforest[9] If the news is the same, the elements that mostdestruction

people focus on should be identical.

was

9,166

squarekilometers,

marking an 85% increase from the previous year.
The Association for College and ResearchThis sharp increase in deforestation coincides

Libraries (ACRL) defines information literacy aswith a period when the current President of“a set of abilities requiring individuals to

Brazil relaxed restrictions on the development of‘recognize when information is needed and havethe Amazon rainforest, after assuming office.”

the ability to locate, evaluate, and use effectively

From this news, select two statements we canthe needed information.’” (ACRL 2000: 2). Basedconfidently make.

on this definition from the ACRL, Podgornik et[1] The area of the Amazon rainforest

al. (2016) devised a test-style questionnaire. We

destroyed in 2019 was larger than in 2018, [2]arranged it specifically for Japanese respondents.

The current President of Brazil orchestrated thedestruction of the Amazon rainforest, [3] The(1) From the following options, select one

destruction of the Amazon rainforest significantly

piece of unprocessed, raw data:

impacts climate change, [4] The current[1] The number of UN member states at the

President of Brazil relaxed regulations on theend of 2020, [2] Weather map, [3] Population data

development of the Amazon rainforest, [5] The

presented in a table, [4] Population data

destruction of the Amazon rainforest emerged as

represented in graphs

a major issue for the first time in 2019, [6] One of
the significant global issues is the destruction of(2) Which of the following statements about

the Amazon rainforest.

restaurant X does not include the author’s

Control variables

opinion? Select one.

In addition to the demographic characteristics,[1] Restaurant X is situated far from the station,
40

Journal of Socio-Informatics Vol. 16 No. 1 Sep. 2023

we used trust in the media as a control variable.

checking results.

This is predicated on the assumption thatThe

secondvariablewaswhethertheindividuals with higher levels of trust in a given

respondents were misled after fact-checking. A

medium are more likely to accept fact-checkbinary variable was created by assigning ‘1’ toresults presented through that medium. We

those who chose ‘1’, and ‘0’ to those who selected

established four variables corresponding to the‘2’ or ‘3’ from the given options. For this analysis,four media categories utilized in this study: online

the entire sample was included because those

news, social media, mass media, and government

who

websites. We assessed trust in each of these

subsequently changed their minds after fact-

media types using a five-point scale.

checking, thus leading to incorrect responses,

initially

respondedcorrectly

butshould be considered part of the ‘adverse effectDependent variablesof fact-checking’.

As previously mentioned, we established two

The basic statistics of the variables are listed in

types of dependent variables. The first determined

Table 5.

whether the respondents achieved accurate

Modelperception after fact-checking. A binary variable

The two following models were developed:

was created by assigning a ‘1’ to those who chose‘2’ and a ‘0’ to those who selected ‘1’ or ‘3’ from（－

logit［P（Trueij＝1）］＝log

the three options: “1 = It is correct information,”“2 = It is misinformation / unsubstantiated1=α+β1Charactristicsij+

information,” and “3 = Do not know / cannot say）

P［Trueij］
P［Trueij］

β2 Media_dummyi+β3 Literacyi+β4Trusti

either.” It should be noted that those whodetected misinformation prior to fact-checking
were excluded from the analysis, as the research

… (1)

（－

logit［P（Falesij＝1）］＝log

question appropriately targets those whose
perceptions changed after receiving the fact-1=α+β1Charactristicsij+
β2 Media_dummyi+β3 Literacyi+β4Trusti

Table 5 Basic Statistics

41）

P［Falesij］
P［Falesij］

（－）

P［Fales ］
P［Falesij］

ij
Literacyis（necessary
to understandlogit
Falesij＝1）［P］＝log Fact-Checking: An empirical research using survey experiments

1=α+β1Charactristicsij+

TANIHARA et al

degree to which individual i trusts each medium

β2 Media_dummyi+β3 Literacyi+β4Trusti… (2)outlet. A vector with four variables— trust in

These are logit models. The right-handgovernment websites. Falseij is a dummy variablevariables were standardized, excluding dummy

indicating whether individual i misidentified

variables. Stata ver. 16.1 was used in the analysis.

misinformation as correct information about topic

Each variable has the following meaning: Trueij is

j despite it being fact-checked. It is set to 1 if

a dummy variable indicating whether individual i

misidentified and 0 if not.

online news, social media, mass media, andwas able to identify the news as misinformation

4. Results

after being fact-checked for topic j. It is 1 if correct,
and 0 if not. P(Trueij=1) is the probability that
Trueij. Charactristicsij is the attribute vector of

Table 6 details the results of Equation (1).

individual i. Specifically, it is a vector with three

Regarding political misinformation, no variations

variables: gender, age, and interest in the topic.

were detected between media conveying fact-

For interest in the topic, we asked about thechecking,

degree of interest on a five-point scale for bothsignificantly positively correlated. For COVID-

topics. Media_dummyi is a dummy variable

19-related misinformation, government websites

indicating that media individual i received fact-

significantly positively impacted the effects,

checking. Social media was used as a reference.

indicating

individual

higherlikelihood

literacyofwascorrect

perception when fact-checking results wereSpecifically, it is a vector with two variables: media

presented on this source compared to social

and information literacy. Trusti indicates themedia. Both media and information literacy

i’sliteracyainformation

vector.

Literacyi

is

while

Table 6 Results of Logit Model Analyses (1)42Journal of Socio-Informatics Vol. 16 No. 1 Sep. 2023

Table 7 Results of Logit Model Analyses (2)exhibited positive associations.

effective one in identifying misinformation. This

Table 7 lists the results for Equation (2),consistency is fitting as accepting fact-checking

identifying characteristics of individuals who

results essentially involves discerning truth fromcould not identify misinformation despite being

falsehood. Information literacy, which includes

presented with fact-checking results. For politicaldifferentiating facts from opinions and proficientmisinformation,

textual comprehension, proves crucial in suchnovariations

were

noted

between the media conveying fact-checking.

judgments.

However, both media literacy and information

significance of information literacy, suggesting

literacy had significant negative correlations. For

that a more literate public is less susceptible to

COVID-19-related misinformation, the medium

the detrimental impacts of misinformation—aof

key insight for policymakers.

newspapers

significantly

reducedthelikelihood of misperceptions compared to social

This

study

underscores

theSecondly, media literacy fosters the acceptancemedia. Both media and information literacy wereof

negatively associated.

misinterpretation, barring political news infact-checking

resultsandhinders

Equation (1). Despite inconsistent results in

5. Discussion

earlier studies (Vraga et al., 2020; Tanihara et al.,2022), the impact of media literacy became

Our study primarily reveals that information

clearer when measured in a test format.

literacy facilitates the acceptance of fact-checking

As detailed above, literacy fosters acceptanceresults and mitigates misinterpretation. Amongofthe four literacies presented by Jones-Jangmisinterpretation.

(2021), information literacy stands as the soledeveloped over a medium-to-long-term horizon.
43

fact-checking

resultsThese

andskillsmitigates
shouldbeLiteracy is necessary to understand Fact-Checking: An empirical research using survey experiments

TANIHARA et al

Media literacy is the ability to critically interpret

checking results. Although this phenomenon

broadcasted information. In the contemporary

doesn’t align with the scope of the backfire effect,era, where information is disseminated throughour data demonstrate its occurrence. This

diverse media such as websites and social media,

circumstancein addition to traditional outlets like television

policymakers. Once again, media and informationand newspapers, media literacy has become anliteracy emerge as the solutions. This study

increasingly crucial skill. Concurrently, fact-

unequivocally establishes the vital role of both

checkers should recognize that fact-checking

types of literacy in facilitating the societal

results should be subject to critical analysis.

penetration of effective fact-checking.

Moreover, fact-checking results should bechallenge

formedia on fact-checking acceptance. For news

Subsequently, we offer an interpretation of theto

COVID-19,

groupsexposedtogovernment websites were more likely to discern

Recent empirical studies suggest that the backfire

the truth than those exposed to social media.

effect is limited (Wood and Porter, 2018; Swire et

Additionally, the group exposed to newspapersal., 2017) and is observed chiefly when targeted

was less likely to misinterpret information thaninformation and participants exhibit strong

those exposed to social media. These observations

partisanship (Wood and Porter, 2018). The news

underline the inherent vulnerabilities of social

discussed in this study, the abolition of plastic

media as an information medium, as pointed out

bags by the ruling and foreign governments’by Tanihara et al. (2022). In essence, social media

vaccine enforcement for children, may stir strong

has certain limitations as a medium for fact-

partisanship, particularly the latter. Yet, thechecking dissemination. To effectively disseminate

former topic doesn’t incite a high degree offact-checking results, it is necessary to develop acontroversy. Regression analysis results indicate

reliable platform and systematize the dissemination

that, irrespective of the partisanship of specificprocess. These findings underscore the risk

news, individuals with low information literacy

associated with casually broadcasting fact-checkare particularly susceptible to the backfire effect.

results on social media. If the recipients lack

While related studies have considered individual

literacy, the outcomes could be counterproductive.

beliefs

asofrelatedfact-checking.

political

effectsa

The third issue concerns the impact of various

ingeniously disseminated.
counterproductive

presents

explanatory

variables,

6. Contributions and limitations

addressing this policy-wise is challenging.
Contrarily, this study identifies information
literacy, a variable that can be addressed by

The novelty of this study lies in the following:

policy, as the cause of the backfire effect. This

firstly, we revisited the relationship between

presents a substantial contribution to theliteracy and fact-checking, improving the literacy-

discourse, revealing the potential to artificiallymeasuring items. Secondly, we sought to gauge

curtail the backfire effect.

the differential effects of various media byrandomly segmenting the sample, focusing on

The aforementioned observations also apply tothese differences.

individuals who had accurate perceptions orThis study enriches the literature on corrective

suspended judgment before examining fact44

Journal of Socio-Informatics Vol. 16 No. 1 Sep. 2023

information effects. While related research has

online news screens, social media timelines,

primarily centered on individual political beliefs asnewspaper articles, and government website

variables, our study highlights the importance ofscreens, and then solicit respondents’ judgments.

literacyandthedisseminationformediumoffact-checking

Moreover, the research design must acknowledge

effective

fact-checking

that fact-checking is a process. Simply presenting

communication and misperception prevention.

short report results, as done in this study, may

These variables are relatively policy-responsive andfail to encapsulate the essence of fact-checking—ripe for further exploration. Though literacy hasverifying information truthfulness from various

yielded inconsistent outcomes in prior studies, ourperspectives, sharing the process with the public,

findings underscore its significance in counteracting

and leaving the judgment of fact-checking results

misinformation. The study also offers fresh insights

up to the individuals. Capturing this complexity

into the backfire effect. The unfortunate potential

in a survey experiment poses a significant

for fact-checking to mislead individuals with lower

challenge.

literacy represents a new risk in fact-checking that

Acknowledgments

requires acknowledgement.
The policy implications of our study are as

The survey was conducted as part of a researchfollows: information literacy is crucial for bothproject by the GLOCOM. This work was also

political and COVID-19-related news, as it is

supported by JSPS (Japan Society for the

integral to misinformation counteraction. The

Promotion of Science) KAKENHI Grant Number

ability to discern between facts and opinions, andJP21K12586 and Google Japan.

to read and comprehend texts—components of
information literacy used in our study—should

References

be stressed in educational curricula. Concerning

Allcott, H., & Matthew, G. (2017) Social media andmedia, only COVID-19-related news was found to

fake news in the 2016 election. Journal of Economic

be effectively disseminated via government

Perspectives 31(2), pp.211-236.

websites. Particularly considering the ‘adverse

Association for College and Research Libraries (ACRL).

effect of fact-checking,’ hastily disseminating(2000) Information literacy competency standards for

information on social media carries the risk ofhigher education. Chicago IL, American Librarymisinterpretations. It is therefore important to

Association.

convey fact-checking results through accredited

informationliteracycompetency. (Retrieved on May

media.

http://www.ala.org/acrl/standards/

5, 2022)

This study is not without its limitations,

Jones-Jang, S.M., Mortensen, T., & Liu, J. (2021)

particularly in the design of the survey

Does media literacy help identification of fakeexperiment used to discern different media

news? Information literacy helps, but othereffects. In the survey, all groups were divided

literacies don’t. American Behavioral Scientist 65

based on the medium at the sentence level—an(2), pp.371-388.

assumption underlying our experiment. For a

Kodera, A. (2017) Research on the creation of amore eraborated analysis of different media

media literacy measurement scale. Toyo Eiwa

effects, it would be necessary to present actual

journal of the humanities and social sciences 34(3),45Literacy is necessary to understand Fact-Checking: An empirical research using survey experiments
pp.89-106. (in Japanese)

TANIHARA et al

Tanihara, T, Yamaguchi, S., Watanabe, T, & Oshima.

Lewandowsky, S., Ecker, U.K.H., Seifert, C., M.,

H. (2022) Effects of corrections on COVID-19-

Schwarz, N., & Cook. J. (2012) Misinformation

related misinformation: Cross-media empiricaland its correction: Continued influence andanalyses in Japan. International Journal of Web

successful debiasing. Psychological Science in theBased Communities 18(1), pp.41-63.

Public Interest 13(3), pp.106-131.

Vraga, E.K., Bode, L., & Tully, M. (2022). Creating

McLuhan, M. (1964) Understanding media: thenews literacy messages to enhance expert

extensions of man, McGraw-Hill.

correctionsofmisinformationon

Twitter.

Ministry of Internal Affairs and Communications.

Communication Research 49(2), pp.245–267.

(2020) An information distribution survey on new

Walter, N., Cohen, J., Holbert, R.L., & Morag, Y.

coronavirus infections. https://www.soumu.go.jp/(2020). Fact-checking: A meta-analysis of what

menu_news/s-news/01kiban18_01000082.html.

works and for whom. Political Communication(in Japanese) (Retrieved on May 5, 2022)

37(3), pp.350-375.

Nyhan, B. & Reifler, J. (2010) When corrections fail:

Walter, N., & Murphy, S.T. (2018). How to unring the

The persistence of political misperceptions.

bell: A meta-analytic approach to correction of

Political Behavior 32, pp.303-330.

misinformation.

Nyhan, B., & Reifler, J. (2012) Misinformation andCommunication

Monographs,

85(3), pp.423-441.

fact‐checking: Research findings from social

Wood, T., & Porter, E. (2019). The elusive backfire

science. New America Foundation Media Policy

effect: mass attitudes’ steadfast factual adherence.

Initiative Research Paper. Washington, DC.

Political Behavior 41, pp.135-163.

Shin, J., & Thorson, K. (2017) Partisan selective sharing:

Yamaguchi, S., Kikuchi, E., Tanihara, T., & Oshima,

The biased diffusion of fact-checking messages on

H. (2020). Information environment and socialsocial media. Journal of Communication, 67(2),coping in the age of fake news with COVID-19,

pp.233-255.

Innovation Nippon 2020 Report, http://www.

Swire, B., Berinsky, A.J., Lewandowsky, S., & Ecker,

innovation-nippon.jp/. (in Japanese) (Retrieved on

U. K. H. (2017) Processing political misinformation:

May 5, 2022)

Comprehending the Trump phenomenon. Royal
Society Open Science 4 (3), 160802.

46