The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Who
Authors: Michael Sejr Schlichtkrull
Nedjma OUSIDHOUM
Andreas Vlachos
Venue: EMNLP 2023 Findings (2023)
Keywords: fact-checking
automated fact-checking
content analysis
intended use
natural language processing
artefacts
TL;DR: We find that many papers on automated fact-checking are too vague when discussing intended use, and give recommendations to clarify discussion

Abstract: Automated fact-checking is often presented as an epistemic tool that fact-checkers, social media consumers, and other stakeholders can use to fight misinformation. Nevertheless, few papers thoroughly discuss \textit{how}. We document this by analysing 100 highly-cited papers, and annotating epistemic elements related to intended use, i.e.,\ means, ends, and stakeholders. We find that narratives leaving out some of these aspects are common, that many papers propose inconsistent means and ends, and that the feasibility of suggested strategies rarely has empirical backing. We argue that this vagueness actively hinders the technology from reaching its goals, as it encourages overclaiming, limits criticism, and prevents stakeholder feedback. Accordingly, we provide several recommendations for thinking and writing about the use of fact-checking artefacts.

The Intended Uses of Automated Fact- Pact-Checking Artefacts: Why, How and Who
Michael Schlichtkrull' Nedjma Ousidhoum 1.2, Andreas Vlachos
DDpaarteent of Computer Science and Technology, University of Cambridge, "Cardiff University m 4 cam. ac. uk OusidhoumNe cardiieaaarr...... ac.uk av3080cam, ac. uk
Abstract
In this past election cycle for the 45th Pres- ident of the United States, the world has witnessed a growing epidemic of fake news. The plague of fake news not only poses serious thre- ats to the integrity of journalism, but has also creat- nurmoils in the political world. The worst real- world impact [...]
Automated fact-checking is often presented as an epistemic tool that fact-checkers, social me- dia consumers, and other stakeholders can use to fight misinformation. Nevertheless, few pa- pers thoroughly discuss how. We document this by analysing 100 highly-cited papers, and notating epistemic elements related to intended use, i.e., means, ends, and stakeholders. We find that narratives leaving out some of these aspects are common, that many papers propose inconsistent means and ends, and that the fea- sibility of suggested strategies rarely has em- pirical backing. We argue that this vagueness actively hinders the technology from reaching its goals, as it encourages overclaiming, limits criticism, and prevents stakeholder feedback. Accordingly, provide several recommenda- tions for thinking and writing about the use of fact-checking artefacts.
Vlachos and Riedel (2014) the first to lease a public fake news detection and fact-checking dataset, it only includes 221 statements, which not permit machine learning based assess- ments.
To address these issues, introduce the
LIAR dataset, which includes 12,836 short staten- ents labeled for truthfunness, subject, context/venue, speaker, state, party, and prior history.
Figure I: Annotated quotes from Wang (2017). We high- light the motivation of the work in blue and the model means (classification) in yellow The goal stated (epis- temic ends) is to limit misintormation. However, the authors do not state the data actors (who) and the appli- cation means (how) for reaching this goal.
1 Introduction
Following an increased public interest in online misinformation and ways to fight it, fact-checking has become indispensable (Amold, 2020). This has been matched by a corresponding surge in NLP work tackling automated fact-checking and related tasks, such rumour detection and de- ception detection (Guo et al., 2022). In the process, many models, datasets, and applications have been ereated, referred to as artefacts. key part of technological artefacts are their intended uses (Hilpinen, 2008; Kroes and van de Poel, 201 4).. Some have argued that artefacts can only be fully understood through these (Krippendorff., 1989). Spinoza (1660) argued that hatchet which does not work for its intended purpose chopping wood longer hatchet. Heidegger (1927) went further, arguing that artefacts can only be properly understood when actively used for their intended purpose; i.c., the only way to understand a hammer is to hammer with it.
Automated fact-checking artefacts are no differ- ent. The majority of papers envision them as epis- temic tools to limit misinformation. In our analysis, we find that 82 out of 100 automated fact-checking papers are motivated as such. Unfortunately, many papers only discuss how this will be achieved in vague terms authors argue that automated fact- checking will be used against misinformation, but not how or by whom (see Figure 1).
Connecting research to potential use allows re- scarchers to shape their work in ways that take into account the expressed needs of key stakeholders, such as professional fact-checkers (Nakov et al., 2021). It also enables critical work (Haraway, 1988), and facilitates thinking about unintended shortcomings, e.g., dual use (Leins et al., 2020;

---

We find that many papers contain narratives with similar types of vagueness, leaving out key details such as the actors involved. Based on this, we give recommendations to clarify discussions of in- tended use: 1) clearly state data actors, 2) clearly state application means, 3) account for stakehold- ers beyond journalists, 4) ensure that the epistemic narratives are coherent and supported by relevant literature, and 5) empirically study the effective- ness of the proposed uses.
Model owner
osee
Application means
produce
Model Meana
Data subjects
Data actors
Figure 2: Diagram of epistemic elements in automated fact-checking narratives. For example, "journalists (data actors) should use classification model (model- ing means) owned by media company (model owner) to triage claims (application means) made by politicians (ends)."
Related Work
A series of recent studies investigate the values and practices of NLP and machine learning research. Birhane (2021) argues that current practices in ma- chine learning often follow historically and socially held norms, conventions, and stereotypes when they make predictions. Birhane et al. (2022) fur- ther study the values encoded in ML research by annotating influential papers, finding that, while re- search is often framed as neutral, resulting artefacts can deeply affect institutions and communities.
(data subjects), in order to limil This constitutes a narrative, the efficacy of which may have feasibility support (e.g., user study).
Kaffee et al., 2023). Grodzinsky et al. (2012) ar- gue that "people who knowingly design, develop, deploy, use a computing artifact do re- sponsibly only when they make reasonable effort take into account the sociotechnical systems in which the artifact is embedded". Underspecifica- tion risks overclaiming, as authors cannot estimate if the aim they claim to work towards can actually be met by the means they employ.
Several papers have recently examined assis- tive uses of NLP. Buçinca al. (2021) found that users frequently "overrely" on AI/ML tools, accept- ing suggestions even when they are wrong with- out double-checking. Consequently, Perry et al. (2022) found that coding assistants can lead to in- secure code. However, Vasconcelos et al. (2023) showed that explanations could reduce this effect- albeit only long the explanations are designed to be easily understandable and verifiable by the user. Similar our criticism of research in auto- mated fact-checking, Gooding (2022) argued that, for text simplification, the disconnect between algo- rithms and their applications hindered discussions of ethics.
For automated fact-checking this is especially problematic. Overclaiming can create misinforma- tion about misinformation, resulting in the opposite effect from the one intended (Altay et al., 2023). We put forward the notion that the development of automated fact-checking into a useful tool for fighting misinformation will be more constructive with clearer vision of intended uses.
In this paper, we investigate narratives about intended use in automated fact-checking papers. We focus on those where fact-checking artefacts introduced as epistemic tools, i.e., where the aim is increase the quality or quantity of knowledge possessed by people. We analyse 100 highly cited papers on the topic. We manually annotate the epistemic elements of their narratives (shown in Figure 2), relying on content analysis (Bengtsson, 2016; Krippendorff, 2018). 8). Specifically, we extract the stakcholders whose epistemic experience will be affected (i.e., data subjects and data actors), the strategies the authors propose (i.c., application and model means), and their intended goals (i.e., ends).
In automated fact-checking, Nakov et al. (2021); Konstantinovskiy et al. (2021) have recently at- tempted to connect research users by examining the concerns of professional fact-checkers. Gener- ally, they find that organisations desire automation beyond veracity prediction models, which cannot function in isolation of media monitoring, claim detection, and evidence retrieval. Along parallel lines, Glockner al. (2022) found that many au- tomated fact-checking datasets make unrealistic assumptions about what information is available at inference time, limiting the usefulness of the resulting artefacts (i.c., systems and datasets) real-world scenarios.
We categorise the narratives extracted by analysing the links between these elements.

---

modeling means: what machine learning ap- proaches the paper takes (c.g., classification). application means: how the authors want to Search engines have previously been studied as epistemic tools, with findings generalising au- tomated fact-checking. Simpson (2013) argued that search engines perform the roles surrogate experts, connecting information seekers to testi- mony, and choosing how to prioritise testimonial authorities. Miller and Record (2013, 2017) further argued that search engine providers, developers, and researchers therefore hold responsibility for beliefs formed by users, as the internal workings of systems are not accessible to users. There are docu- mented real-world consequences from this opacity, e.g., Machill and Beiler (2009) found evidence of bias in journalistic publications due to overreliance on biased ranking algorithms.
apply the model to accomplish the goal (e.g., by triaging claims),
ends: the purpose of the designed system (e.g., limiting misinformation).
We add further level of annotation at the discourse-level that is, spanning and including epistemic elements from multiple quotes. Here, we extract epistemic narratives, i.e., the stories about knowledge told in each paper (e.g., automated con- tent moderation). Each narrative is associated with specific set of data actors (e.g.. journalisss, Nar- ratives can be associated with multiple elements, and elements can be part of multiple narratives. We also extract the feasibility support for each narrative, i.e., any backing for the feasibility or effi- cacy of that narrative (e.g., scientific research). See the illustration in Figure 2 for an overview of our framework.
3 Content Analysis
To understand the narratives of the fact-checking literature, we perform content analysis following Bengtsson (2016); Birhane et al. (2022) by means of annotating 100 highly-cited papers. As recom- mended by Krippendorff (2018), we annotated the data in two rounds, working inductively in the first round. That is, during the first pass we began with a small set of labels for each element, and added new as necessary. We then unified the two sets of labels (i.e., the old and new sets of labels), merging labels where necessary. For the second pass, we re- annotated each paper with labels from the unified list, adding
3.2 Annotation Process
The annotation was conducted by the first two au- thors, NLP experts working on automated fact- checking. After the first annotation round, we clarified the definitions of each category, designed
a flowchart to improve the consistency of the narrative-level annotations (see Appendix B.5), and re-annotated the data. This process yielded a sub- stantial Krippendorff-a agreement score of 0.76 on our narrative annotations, using Jaccard distance as the metric. We provide the list of papers and full of annotations at hztps : / //gthubb. com/ mi chSch1 1 PottendddaFFUseess and the full anno- tation guidelines in Appendix B.
labels.
3.1 Annotation Scheme
We start our analysis by extracting quotes from the introductory sections of papers. These are spans starting and ending at sentence boundarics (although potentially spanning multiple sentences within a paragraph). We identify for further pro- cessing those that either discuss what paper does why the authors focus it. Then, we annotate them with epistemic elements. This annotation is labti-label, i.c., for each element, one quote may have several (or no) labels. The elements (in bold) can be seen here along with an example label (in italic):
We collected the papers for annotation using the continually updated repository of papers! focusing on fact-checking and related tasks (e.g., rumor de- tection, deception detection), which accompanies the recent survey by Guo et al. (2022). To focus our investigation on influential narratives, we limited our analysis to the 100 most cited papers listed in the repository. We manually extracted quotes from the introduction sections of each paper, and when important elements were missing, we additionally considered quotes from the abstract.
data subjects: the people whose behaviours are analysed (c.g., social media users),
data actors: the people who are intended to use the model outputs (e.g.. journalist), model owners: peoplelorganisations who own the model and may make choices about its deployment (e.g. social media companies),
'nttps: thtp:////gtthhhoocce wttp:/attc/////ccc////oo Aut.omated-Fach

---

4 Extracted Epistemic Elements
"The dissemination of fake nevs may cause large-scale negative effects, and sometines can affect or even manipulate important public events. [..] Therefore, it is in great need of un automatic detector to mitigate the serious negative effects caused by the fake news." vague identification in Wang et al. (2018).
We extracted the following lists of labels for epis- temic elements, along with the percentage of narra- tives they appeared in. Note that the annotation is multilabel, so the percentages do not sum to 1. Defi- nitions of each label can be found in Appendix B.2.
"The ever inerreecceaiingg amounts of textual information available combined with the ease in sharing it through the web has increased the demand for verification, also referred to fact checking. [...] In this paper.
Data Subjects Social media users (24.9%), pro- fessional journalists (15%), politicians & public figures (5.6%), product reviewers (1.7%). technical writers (1.7%), citizen journalists (0.4%).
introduce new dataset..."
vague opposition in Thorne et al. (2018).
" "Rumours rife the web. Folse claims affect peo- plc's perceptions of events and their behaviour, times in harmful ways [...] While breaking news un- fold, gathering opinions and evidence from many sources possible as communities react becomes crucial determine the verucity of runours and sequently reduce the impact of the spread misinfor- mation." "
Data Actors Professional journalists (18.5%) citizen journalists (2.6%). media consumers (4.7%).
scientists (3.9% (3.9)), algorithms (1.7%), engineers & curators (1.3%), law enforcement (0.9%).
vague debunking in Derczynski et al. (2017).
Model Owners Social media companies (4.7%). scientists (1.3%), law enforcement (0.9%), tradi- tional media companies (0.4%).
Figure Examples of vague narratives highly cited automated fact-checking papers.
Modeling means Classify/score veracity (79.3%), evidence retrieval (24.5%), produce justifications (6.9%), human in the loop (11.2%), classify/score stance (3.9%) corpora analysis (2.1%), generate claims (0.9%)
5.1 Vague narratives
We find that the majority of papers, 56%, contextu- alize their research through narratives we charac- terize as vague, which make it difficult to reason about or criticize the potential applications of the proposed technologies. We identify three primary "types" of vagueness, discussed below. See Figure 3 for example of each.
Application means Identify claims (26.2%) pro- vide veracity scores (17.5%), supplant human fact- checkers (13.7%) gather and present evidence (10.7%), present aggregates of social media com- ments (6.9%). triage claims (6.0%), vague persua- sion (3.4%)..fol filter system outputs (3.0%), analyse data (3.0%), automated removal (1.7%), identify multimodal inconsistencies (1.7%), maintain con- sistency with knowledge base (1.3%), produce misinformation (0.9%).
Vague identification (31%) The most common form of vagueness we identify is a narrative where automated fact-checking will be used to identify po- tential misinformation. However, no discussion is made of what will happen to potential misinforma- tion after flagging it. As many options are available automated removal, warning labels, additional work by professional journalists it is difficult to assess the impact of these applications.
Ends Limit misinformation (78. 17),%)) increase veracity of published content (7.2%), limit Al- generated misinformation (3.4%) develop knowl- edge of NLP/language (3.4%) detect falsehood for law enforcement (0.9%), avoid biases of human fact-checkers (0.4%).
Vague opposition (19%) Automated fact- checking is presented as a tool to fight misinfor- mation with no discussion of how that tool will actually be used. Crucially, no application means are mentioned, and other than the final goal opposition to misinformation the intended uses of the artefacts introduced are not discussed.
5 Extracted Narratives
At the discourse level, we extracted epistemic nar- ratives. Below, give short definition of each. We also calculated the percentage of papers each narrative appears in see Figure 4. Note that this annotation is also multilabel, so the percentages do not sum to 1.
Vague debunking (14%) Authors clearly intend their artefacts to aid in the production of evidence- based refutations, i.e., debunking, similar to the

---

function of professional fact-checkers. However. it is not clear whether the system is intended to replace human fact-checkers, or to assist them. data actors are mentioned. The application means are typically to present evidence or veracity scores to users (45% and 40% of cases, respectively).
30
25
Frequency
20
18%
5.2 Clear Narratives
1.1%
addition the vague narratives, we furthermore identify several clearly stated intended uses. We also calculated the percentage of papers each nar- rative appears in. As previously stated, a paper can have many narratives, so the percentages do not sum to 1.
10
8%
8%
Autonated Extenal Bact-cleckinn
Vague idontification
Assisted Estomal eet-heccing
Vague opposition
Assisted media concuption
Vague Debunking
Amissed Knowledge Coration
Solentiic conosit
Assisted internal fact chocking
Autonuted coment moderaiion
Law enforcmmen
Automated external fact-checking (22%) Arte- facts developed in the paper are intended to fully au- tomate the fact-checking process, without human- in- -n-the-loop interventions.
Assisted external fact-checking (18%) Arte- facts are intended as assistive tools for professional or citizen journalists, deployed for external fact- checking (i.c., assistance in writing fact-checking articles).
Figure Frequency each epistemic narrative found in our analysis of 100 automated fact-checking papers.
Findings
Assisted media consumption (8% Artefacts are intended as assistive tools for consumers of information, either as layer adding extra informa- tion to other media or as standalone site where claims can be tested.
Based on our content analysis, identify several areas of concern in the analysed papers. In the following sections, discuss our main findings.
6.1 Stakeholders
Scientific curiosity (8%) Artefacts in the paper are not intended for use. Instead, the process of de- velopment itself is claimed to produce knowledge i.e., about language or misinformation.
Technological artefacts are components in so- ciotechnical systems involving agents (van de Poel, 2020). A crucial consideration is therefore the stakeholders who will be involved in deployment: data subjects, data actors, and model owners.
Assisted knowledge curation (7%) Artefacts are intended as assistive tools for the curation of large knowledge stores, such as Wikipedia.
Who are the fact-checking artefacts for? A key factor of vague narratives is the omission of data tors. For clear narratives, find mentions of data actors in 51% of cases. For vague narratives, this number is dramatically lower: 6.7% for vague op- position, 7.1% % for vague identification, and 0% for vague debunking. Model owners are discussed 10.2% of non-vague narratives, and 2.8% of vague narratives. Intuitively, this is reasonable: thinking about who will use technology also encourages authors to think about how it will be used. Miss- ing data actors is especially problematic for work that seeks to provide explainability, where expla- nations can be understood differently by different users (Schuff et al., 2022). We find that only a small minority, 6.25%, of narratives that include
Assisted internal fact-checking (4%) Artefacts are intended as an assistive tool for professional or citizen journalists, deployed for internal fact- checking (i.c., assistance in fact-checking articles before publication to ensure quality).
Automated content moderation (4%) Artefacts are intended to complement or replace human mod- erators on e.g., social media sites, by performing moderation functions autonomously.
Truth-telling for law enforcement (1%) Arte- facts are intended for use by law-enforcemennt groups or in courtrooms as lie detectors.

---

justification production include any discussion of data actors (professional journalists, in all cases).
6.3 Unsupported ends
We frequently find disconnect between applica- tion means and envisioned ends. For example, 50% of narratives of automated content moderation sug- gest direct, algorithmic removal of items classified as false. This is of questionable effectiveness. Re- moval can induce an increase in the spread of the same message on other platforms, or increase toxi- city (Sanderson et al., 2021; Ribeiro et al., 2021). Censorship can also amplify misinformation via the "Streisand effect", where the attempt to hide in- formation encourages people seek it out (Jansen, 2015; The Royal Society, 2022). Moreover, algo- rithmic bias (Noble, 2018; Weidinger al., 2022) or misuse by model owners (Teo, 2021) could fur- ther distort what is removed. Given that, we find it likely that automated removal would fail either by not reducing belief in misinformation, or by re- ducing belief in other true information. This is not isolated to single narrative. For assisted external fact-checking, several papers suggest providing racity labels or scores to human fact-checkers, who have expressed strong skepticism at the usefulness of such scores (Nakov et al., 2021). We provide further analysis in Appendix A.
And who are we fact-checking, anyway? A concern in AI ethics is the lack of attention paid data subjects, i... the people whose data is in- put to AI systems (Kalluri, 2020). We find that data subjects are discussed in 45% of non-vague and 39% of vague narratives. However, actors and subjects are rarely the same entities - there is over- lap only in 8% of non-vague narratives, and no vague narratives. Further, actors tend to be experts (professional journalists, in 62% of all narratives), rather than peers of the data subjects (social media users, in 65% of narratives). This mirrors Kalluri (2020), who argues that "researchers Al over- whelmingly focus on providing highly accurate in- formation to decision makers. Remarkably little research focuses on serving data subjects."
What about other uses? Combating misinforma- tion is by far the most common end. One narrative has a slightly different focus, assisted knowledge curation, where the aim in 84% of cases is to pre- vent errors in published content. 7% of papers envision applications to curation, including for re- lational knowledge bases well for Wikipedia. The typical application means is to filter the output of other systems, e.g., knowledge base comple- tion models. Survey papers, e.g., Thorne and Vla- chos (2018); Lazer et al. (2018); Kotonya and Toni (2020); Hardalov et al. (2022); Guo et al. (2022) tend not to mention this potential use, even though they go into detail with others.
6.4 Lack of feasibility support
Papers rarely provide evidence in support of the efficacy of their narratives regardless of whether, as with, e.g., assisted media consumption, there is significant work on measured effectiveness (Clay- ton et al., 2020; Pennycook et al., 2020; Sanderson et al., 2021). Only 18% of non-vague narratives, and 0% of vague narratives rely on citations to prior work or relevant litcrature from other fields for this. Furthermore, when narratives are supported by ci- tations, some references do not fully support the claims made. For example, Cohen al. (2011) is cited to support the effectiveness of fully auto- mated faet-checking, but all the original paper does is suggest that such a task might be relevant. That is, there is no underlying quantitative or qualitative data to determine effectiveness.
6.2 Mismatched means
Even in clear narratives, there is often a disconnect between the proposed applications and the actual NLP techniques used. For example, 81% of papers seeking to supplant professional fact-checkers, i.e.. replicate their entire function, rely on classification, and only 31% % include evidence retrieval. High- quality fact-checks primarily focus on finding and explaining available evidence, rather than verac- ity labels (Amazeen, 2015; Konstantinovskiy et al., 2021). Professional fact-checking includes signifi- cant editorial work, and fact-checkers are expected to "never parrot or trust sources" (Silverman, 2013). Instead, fact-checkers reason about the provenance their evidence, and convey that to their readers.
6.5 Evidence is not a silver bullet
The idea that humans should update their beliefs based on model predictions common several narratives. This can be problematic, as model pre- dictions can be inaccurate especially for machine- generated claims that can casily mimic "truthy" styles (Schuster et al., 2020). The commonly pro-

---

posed remedy is to rely on support from retrieved evidence (Thorne et al., 2018; Guo et al., 2022). This view of evidence as a silver bullet is epistemo- logically naive, and high retrieval performance is not enough to provide support for the end of limit- ing misinformation. We identify at least three ways in which retrieving and presenting even credible evidence can still produce epistemic harm.
Knorr Cetina (1999). We would guess, however, that some mix of the following factors could be potential reasons:
Common datasets often either use synthetic, purpose-made claims generated e.g., from Wikipedia, rather than "real" misinforma- tion (Thome et al., 2018), or they do not pro- vide annotations for evidence. Making clear, direct claims about efficacy on real misinfor- mation is therefore difficult.
First, the predictions made by retrieval systems can be self-fulfillig (Birhane, 2021). A system asserting that a particular piece of evidence is more credible or relevant than an alternative can make users agree with that choice, even if they would otherwise not have (Buçinea et al., 2021). This is an existing concern in journalism, known as froogle-izztiinn overreliance on search algo- rithms causes a "distortion of reality" when dead- lines are tight (Machill and Beiler, 2009).
Leaderboarding is casier than including data actors in evaluation (Rogers and Augenstein, 2020), and excluding actors may therefore be the path of least resistance. As an additional result, some authors may see the introduction and the motivation as less important sections than e.g.. experimental results, and put less effort into their writing.
Second, if system presents many retrieved doc- uments without accounting for provenance, users may rely on sheer quantity to make quick (but pos- sibly wrong) judgments about veracity. This is a common bias known as the availability heuris- tic (Schwarz and Jalbert, 2021). A retriever may have collected many documents from one source. or from an epistemic community structured around "rryaa family of knowers" from whom many seemingly independent sources derive their infor- mation (Bala and Goyal, 1998; Zollman, 2007).
To highly invested researchers, the effective- ness of their preferred means may seem obvi- (Koslowski, 2012). Iee,, they may think that "it is self-evident that automatically identifying potential misinformation will help us fight it".
Engaging with the literature in psychol- ogy and journalism on the efficacy of fact- checking may be daunting for NLP re- searchers. Some of this literature may be not be accessible to researchers in other fields, who are nonetheless interested in helping via the automation of the process (Hayes, 1992; Plavén-Sigray et al., 2017).
Finally, evidence can mislead if users hold prej- udices that lead them to make wrongful inferences. Fricker (2007) cites To Kill Mockinghird as an example: because of racial prejudice, evidence that a black man running is interpreted as evidence that he is guilty. Conceivably, a poorly trained or heavily biased fact-checking system could sim- ilarly cause harm by drawing undue attention to irrelevant - but stereotyped evidence.
7 Recommendations
Authors of fact-checking papers clearly believe their research to be solving societal need: 82% of narratives in our analysis had "limiting misinfor- mation" as the desired end, and an additional 7% had "increasing the veracity of published content". As we have argued, vagueness around intended use in current papers may prevent the community from contributing to these goals, as it can block in- put from key stakeholders or obscure discussion of risks. Our recommendations below focus on bring- ing research on automated fact-checking into closer alignment with its stated goal. We emphasise that our intention is not to create another checklist for NLP conferences, as clarity surrounding intended use is already covered in the recently adopted ARR
Unfortunately, these harms tend to affect the already marginalised most (Fricker, 2007; O'Neil, 2016; Noble, 2018). As argued by Birhane (2021), 1), current practices in machine learning often impose order on data by following historically and socially held norms, conventions, and stereotypes".
6.6 Explaining vagueness
It is difficult to speculate on why automated fact- checking papers are prone to vagueness. A proper answer would require interviewing the authors of the papers included in our analysis, i.c., to con- duct ethnographic studies with researchers as in

---

Responsible NLP Checklist (Rogers al., 2021), However, authors may find our recommendations useful for meeting that requirement.
vague opposition narrative. Leaving out applica- tion means disconnects the fact-checking artefacts entirely from any real-world context. Taking Krip- pendorff's (1989) view that technological artefacts must be understood through their uses, this prevents any full understanding of these artefacts. We rec- ommend including clear discussion of application means in any justificatory narrative.
7.1 Discuss data actors
A key component of vague narratives is the ab- sence of data actors. Thinking about who will use technology encourages thinking about how it will be used. As such, first recommendation is to include a clear discussion of the intended users. This also enables input from relevant communi- ties: researchers working on systems designed for professional fact-checkers can seek feedback from those fact-checkers, which is crucial for evaluation. A fact-checking model paired with professional journalist constitutes a very different sociotechni- cal system from one paired with, for example, social media moderator (van de Poel, 2020). If the aim is to evaluate the capacity for technologies accomplish real-world aims, necessary to evaluate them for specific users.
7.3 Account for different stakeholders
Where data actors are mentioned, they are mostly professional journalists (62%). They rarcly overlap with the data subjects, except in the case of internal fact-checking, We note that professional journalists are predominantly white, male, and from wealth- ier backgrounds (Spilsbury, 2017; Lanosga et al., 2017). Misinformation often (1) targets and un- dermines marginalized groups or (2) results from inequality-driven mistrust among the historically marginalised (Jaiswal et al., 2020). Professional journalists may, as such, not be the only actors well-suited for fighting it.
We highlight this especially for systems that produce justifications or explanations, where re- cent work shows that biases and varying techni- cal competence influences what explainees take from model explanations (Schuff et al., 2022). Jus- tifications may be a powerful tool to counteract overreliance on, e.g., warning labels (Pennycook et al., 2020) or search rankings (Machill and Beiler, 2009), but may only be helpful if designed for rele- vant data actors (Vasconcelos et al., 2023).
In the spirit of developing systems for all poten- tial stakeholders, we encourage authors to widen their conception of who systems can be designed for. Media consumers feature prominently as data subjects, but are rarely expected to act on model outputs. This may be a limiting factor for what researchers can accomplish in terms of opposing misinformation. The literature on countermessag- ing highlights the need for ordinary social media users to participate in interventions (Lewandowsky et al., 2020). Tripathy et al. (2010) suggest model where ordinary users intervene against rumours by producing anti-rumours. They argue that this would be superior to traditional strategies, as it limits reliance on credibility signals from distant authorities (Druckman, 2001; Hartman and Weber, 2009; Berinsky, 2017).
We note that avoiding discussion of data actors (or other epistemic elements) can be seen as vari- ant of Haraway's 's (1988) "god trick", where authors avoid situating their research artefacts within the context of real-world uses. This prevents criticism of and accountability for any harm caused by ap- plications of the technology. As actors hold a ma- jor part of the responsibility for the consequences of deploying the technology (Miller and Record, 2017), explicitly including them is necessary to discuss the ethics of use.
Early work on automated fact-checking (Vla- chos and Riedel, 2014) proposed the technology as a tool to enable ordinary citizens to do fact- checking - i.e., citizen journalism. Decentralised, community-driven fact-checking has been shown to represent viable, scalable alternative to pro- fessional fact-checking (Saced et al., 2022; Righes et al., 2023). Citizen journalists have unfortunately largely disappeared since then, appearing as actors only in three narratives (10%) of assisted external fact-checking and one narrative (11%) of assisted internal fact-checking. Similarly, human content
7.2 Discuss application means
Beyond the absence of who is intended to use au- tomated fact-checking artefacts, some papers also lack discussion of how they will be used. That is, the narrativss do not have clear application means. This type of vagueness is particular to the
hnttps://acl ra ingge// "esponsiibleMiiPreeeerra

---

moderators are entirely missing. Data curators, al- though somewhat frequent in research, are often overlooked survey papers. We suggest that study- ing the user needs of groups beyond journalists and social media companies is a fruitful area for future research. Further, it may be relevant to study who spreads misinformation (Mu and Aletras, 2020).
increased accuracy of crowdworkers' veracity judg- ments when presented with additional evidence briefs from their system. This provides evidence for the effectiveness of their suggested narrative (assisted media consumption). Similar investiga- tions were also done in Draws et al. (2022), where the authors explored crowdworker biases in fact- checking. Fruitful comparisons can also be made to evaluations of the effectiveness of explainability techniques (Vasconcelos et al., 2023), persuasive dialog agents (Tan et al., 2016; Wang al., 2019; Altay et al., 2021; Farag et al., 2022; Brand and Stafford, 2022), and code assistants Perry et al. (2022). For fact-checking, this research should be informed by best practices on evaluating hu- man interventions against misinformation (Guay et al., 2023), well as best practises for human evaluations of complex NLP systems (van der Lee et al., 2019). Further, this research should carefully consider epistemic harms of the sort discussed Section 6.5 before claiming superior performance.
7.4 Ensure Narrative Coherence
In our analysis, we identified two unfortunate trends: inconsistencies between modeling and ap- plication means (see Section 6.2), and inconsis- tencies between means and ends (see Section 6.3). These are problematic, as they may in effect be overclaims, c.g., authors claim that a classifier will tackle a task that is not a classification task. In Sec- tion 6.4, we found that only minority of papers draw on the literature to support the feasibility of their proposed narratives. There is significant litera- ture interventions against misinformation by hu- man fact-checkers, For example, Guess and Nyhan (2018); Vraga et (2020); Walter and Tukachin- sky (2020) find evidence for the effectiveness of providing countermessaging: Lewandowsky et al. (2020) find that the countermessaging should ex- plain how the misinformation came about, not just explain why the misinformation is false; and Druck- man (2001); Hartman and Weber (2009); Berinsky (2017) find that the perceived credibility and affil- iation of the countermessages matter great deal. We recommend relying on such literature as a start- ing point to document the coherence of the chosen modeling means, application means, and ends.
8 Conclusion
Researchers working on automated fact-checking envision an epistemic tool for the fight against mis- information. We investigate narratives of intended use in fact-checking papers, finding that the major- ity describe how this tool will function in vague unrealistic terms. This limits discussion of possible harms, prevents researchers from taking feedback from relevant stakeholders into account, and ulti- matcly works against the possibility of using this technology to meet the stated goals.
7.5 Study effectiveness of chosen means
Along with our analysis, we give five recom- mendations to help researchers avoid vagueness. We emphasise the need for clearly discussing data actors and application means, including groups be- yond professional journalists as stakeholders, and ensuring that narratives are coherent and backed by relevant literature or empirical results. Although we have focused on automated fact-checking in this paper, we hope our recommendations can inspire similar introspection in related fields, such as hate speech detection.
Our final recommendation is a call to action on re- search into the effectiveness of various automated fact-checking strategies. We have recommended relying relevant literature to support connec- tions between means and ends. However, for many proposed strategies, there is little or no literature. This represents a problem: if authors intend claims like "our system can help fight misinformation" to be scientific, they should provide (quantitative or qualitative) evidence for those claims. Where no prior literature exists on the efficacy of the pro- posed approach, authors can either rely on the expressed needs of people in their desired target groups (Nakov et al., 2021), gather empirical evidence using sample from the target group.
Limitations
We extracted quotes only from the introductions and abstracts of the papers, following past find- ings that stories of intended use mostly occur in those sections (Birhane et al., 2022). Furthermore, we limited ourselves to clearly stated epistemic
We highlight here the approach taken by Fan et al. (2020), where the authors demonstrated an

---

elements, and did not account for any implied state- ment avoid adding a layer of interpretation the quotes. Therefore, we acknowledge the possibility of missing epistemic elements or narratives subtly stated or mentioned in other sections of the papers. In addition, we analyse list of 100 highly cited NLP papers focusing on automated fact-checking and related tasks. This list is non-exhaustive. Therefore, it certainly does not include all the influ- ential papers, especially recent ones, as they may not have accumulated enough citations yet to sur- pass older work, which does represent a bias.
References
Sacha Altay, Manon Berriche, and Alberto Acerbi. 2023, Misinformation misinformation: Conceptual and methodological challenges. Social Media + Society. 9(1 91)) 1221 150412.
Sacha Altay, Anne-Sophie Hacquin, Coralie Chevallier, and Hugo Mercier. 2021. Information delivered by chatbot has positive impact on covid-19 19 vaccines attitudes and intentions. Journal of Experimental Psychology: Applied.
Michelle A, Amazeen. 2015. Revisiting the Epistemol- ogy of Fact-Checking. Critical Review, 1-(1):1-22
We chose to understand automated fact-checking artefacts through their intended uses. This is only one way understand technologies. As pointed out by Klenk (2021) it could be seen as somewhat sim- plistic, since artefacts are often appropriated for uses unforeseen by their designers. Artefacts may also hold very different properties depending on their sociotechnical contexts, i.c., the agents and institutions that may use or limit the use of the arte- fact (van de Poel, 2020). Recent work proposes to instead understand technological artefacts through their affordances (Tollon, 2022), i.e., the actions they enable (regardless of design). Our analysis focuses exclusively on the stated intentions of the authors, potentially limiting our findings. A com- plete analysis of the technology should also include how it is used in practice, e.g., of the documenta- tion produced at companies, government agencies, and other groups that deploy fact-checking arte- facts. We have left this for future work.
Phoebe Amold. 2020. The challenges of online fact checking. Technical report, Full Fact.
Venkatesh Bala and Sanjeev Goyal. 1998. Learning from Neighbours. Review of Economic Studies, 65(3):595-621.
Ramy Baly, Mitra Mohtarami, James Glass. Lluis Mårquez, Alessandro Moschitti, and Preslav Nakov. 2018. Integrating stance detection and fact check- ing in unified corpus. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 21-27, New Orleans, Louisiana. Association for Computational Linguistics.
Mariette Bengtsson. 2016. How to plan and perform qualitative study using content analysis. NursingPlus open, 2:8-14.
Adam J. Berinsky. 2017. Rumors and Health Care Reform: Experiments in Political Misinformation. British Journal of Political Science, 47(2):241-262.
Abeba Birhane. 2021. The impossibility of automating ambiguity. Artificial Life, 27(1):44 61.
10 Ethical Considerations
As this is review paper, we only report on pub- licly available data from the scientific publications shared in the supplementary materials. When re- producing quotes in our annotations, we did not anticipate any issues. However, if an author asks us to take any quotes from their paper down from our repository, we will do so.
Abeha Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2022. The values encoded in machine learning research. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 173-184.
Charlotte O Brand and Tom Stafford. 2022. Using dia- logues increase positive attitudes towards covid-19 19 vaccines in a mnccine-hesitan population. Royal Society Open Science, 9 220366.
11 Acknowledgements
Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To trust or to think: Cognitive forcing functions can reduce overreliance on ai in ai-assisted decision-making. Proc. ACM Hum. -oomp. Inter- act., 5(CSCWI).
We would like to thank Fabio Tollon, David Strohmaier, Hope McGover, and Lucy Lu Wang for their helpful comments, discussions, and feed- back. Michael Schlichtkrull is supported by the ERC grant AVeriTeC (GA 865958), Nedjma Ousid- houm was supported by the EU H2020 grant MO- NITIO (GA 965576) and Andreas Vlachos is sup- ported by both AeeriTeC and MONITIO.
Katherine Clayton, Spencer Blair, Jonathan A. Busam, Samuel Forstner, John Glance, Guy Green, Anna Kawata, Akhila Kovvuri, Jonathan Martin, Evan Mor- gan, Morgan Sandhu, Rachel Sang, Rachel Scholz- Bright, Austin T. Welch, Andrew G. Wolff, Amanda

---

Zhou, and Brendan Nyhan. 2020. Real Solutions for Fake News? Measuring the Effectiveness of General Warnings and Fact-Check Tags in Reducing Belief' in False Stories on Social Media. Political Behavior, 12(4))10733-105.
Frances Grodzinsky, Keith Miller, and Marty J Wolf. 2012. Moral Responsibility for Computing Artifacts: "The Rules" and Issues of Trust. 42(2).
Brian Guay, Adam Berinsky, Gordon Pennycook, and David Rand. 2023. How To Think About Whether Misinformation Interventions Work. PsyArXiv preprint.
Sarah Cohen, Chengkai Li, Jun Yang, and Cong Yu.
2011. Computational journalism: A call to arms to database researchers. pages 148-151. 5th Biennial Conference on Innovative Data Systems Research, CIDR 2011: ; Conference date: 09-01-2011 Through 12-01-2011.
Andrew Guess and Brendan Nyhan. 2018. Selective Exposure to Misinformation: Evidence from the con- sumption of fake news during the 2016 U.S. presi- dential campaign.
Leon Derezynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. 2017. SemEval-2017 task 8: RumourEval: Determining rumour veracity and support for ru- mours. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 69-76, Vancouver, Canada. Association for Computational Linguistics.
Zhijiang Guo, Michael Schlichtkrull, and Andreas Vla- chos. 2022. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10: 178-206.
Donna Haraway. 1988. Situated Knowledges: The Sci- ence Question in Feminism and the Privilege of Par- tial Perspective. Feminist Studies, 14(3):575.
Tim Draws, David La Barbera, Michael Soprano, Kevin Roitero, Davide Ceolin, Alessandro Checco, and Ste- fano Mizzaro. 2022. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Trans- parency, FAceT '22, page 2114-2124, New York, NY, USA. Association for Computing Machinery. [link].
Momchil Hardalov, Amav Arora, Preslav Nakov, and Isabelle Augenstein. 2022. A survey on stance de- tection for mis- and disinformation identification. In Findings of the Association for Computational Lin- guistics; NAACI. 2022, pages 1259-1277, Seattle, United States. Association for Computational Lin- guistics.
James N. Druckman. 2001. On the Limits of Framing Effects: Who Can Frame? The Journal of Polities, 63(4): 1041-106..
Todd K. Hartman and Christopher R. Weber. 2009, Who Said What? The Effects of Source Cues in Issue Frames. Political Behavior, 31 (44:5337-558
Angela Fan, Aleksandra Piktus, Fabio Petroni, Guil- laume Wenzek, Marzieh Sacidi, Andreas Vlachos, Antoine Bordes, and Sebastian Riedel. 2020. Gen- crating fact checking briefs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7 7147-7161, Online. Association for Computational Linguistics.
Donald P Hayes. 1992. The growing inaccessibility of science. Nature, 33666377))7339-741
Martin Heidegger. 1927. Being and Time. John Mac- quarrie, Edward S. Robinson, 1962 (Trans). Harper & Row.
Youmna Farag, Charlotte Brand, Jacopo Amidei, Paul Piwek. Tom Stafford, Svetlana Stoyanchev, and An- dreas Vlachos. 2022. Opening up minds with argu- mentative dialogues. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4569-4582, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Risto Hilpinen. 2008. On artifacts and works of art. Theoria, 58(1):58-82.
J Jaiswal. C LoSchiavo, and DC Perlman. 2020. Dis- information, misinformation and inequality-driver mistrust in the time of covid-19: lessons unlearned from aids denialism. AIDS and Behavior, 24:2776- 2780.
Miranda Fricker. 2007. Epistemic Injustice: Power and the Ethics of Knowing. Oxford University Press.
Sue Curry Jansen. 2015. The Streisand Effect and Cen- sorship Backfire. International Journal of Communi- cation, 16656-671.
Max Glockner, Yufang Hou, and Iryna Gurevych. 2022. Missing counter-evidence renders NLP fact-checking unrealistic for misinformation. In Proceedings of the 2022 Conference on Empirical Methods Nat- ural Language Processing, pages 5916-5936. Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Lucie-Aimée Kaffee, Arnav Arora, Zeerak Talat, and Is- abelle Augenstein. 2023. Thomy roses: Investigating the dual use dilemma in natural language processing.
Pratyusha Kalluri. 2020. Don't ask if artificial intelli- gence is good or fair, ask how it shifts power. Natur, 583(7815):169-1B) 169.
Sian Gooding, 2022. On the ethical considerations of text simplification. In Ninth Workshop on Speech and Language Processing for Assistive Technologies (SLPAT-2022), pages 50-57, Dublin, Ireland, Associ- ation for Computational Linguistics.
Michael Klenk. 2021. How do technological artefacts embody moral values? Philosophy Technology. 34(3):525-544

---

Karin Knorr Cetina. 1999. Epistemic Cultures: How the Sciences Make Knowledge. Harvard University Press.
Marcel Machill and Markus Beiler. 2009. The impor- tance of the internet for journalistic research: A multi- method study of the research performed by journal- ists working for daily newspapers, radio, television and online. Journalism Studies, 10(2):18--00)
Lev Konstantinovskiy, Oliver Price, Mevan Babakar, and Arkaitz Zubiaga. 2021. Toward automated factchecking: Developing an annotation schema and benchmark for consistent automated claim detection. Digital threats: research and practice, 1-2):1-16.
Boaz Miller and Isaac Record. 2013. Justified belief in digital age: on the epistemic implications of secret internet technologies. Episteme, 10(2):117-134
Barbara Koslowski. 2012.
Scientific reasoning: Ex- planation, confirmation bias, and scientific practice. Handbook of the psychology of science, pages 151 192.
Boaz Miller and Isaac Record. 2017. Responsible epis- temic technologies: A social-epistemoloicca anal- ysis of autocompleted web search. New Media & Sociery, 19(12):1945-163
Neema Kotonya and Francesca Toni. 2020. Explain- able automated fact-checking A survey. In Proceed- Com-
Yida Mu and Nikolaos Aletras. 2020. Identifying Twit- ter users who repost unreliable news sources with linguistic information. PeerJ Computer Science, 6xe325.
ings of the 28th International Conference putational Linguistics. pages 5430-5443, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics.
Preslav Nakov, David Corney, Maram Hasanain, Firoj Alam, Tamer Elsayed, Alherto Barrón-Cedeñio, Paolo Papotti, Shaden Shaar, and Giovanni Da San Martino. 2021. Automated fact-checking for assisting human fact-checkers. In Proceedings of the Thirtieth Inter- national Joint Conference on Artificial Intelligence, IJCAI-21, pages 4551-45555 International Joint Con- ferences on Artificial Intelligence Organization. Sur- vey Track.
Klaus Krippendorff. 1989. On the Essential Contexts of Artifacts or on the Proposition That "Design Making Sense (Of Things)". Design Issues, 5(2):9.
Klaus Krippendorff. 2018. Content analysis: An intro duction to its methodology. Sage publications.
PA Kroes and Ibo van de Poel. 2014. Can technol- ogy embody values?. pages 103-124. Springer Sci- ence+Busines, Media.
Safiya Umoja Noble. 2018. How Search Engines Reinforce Racism. NYU Press.
Algorithms of Oppression:
Gerry Lanosga, Lars Willnat, David H Weaver, and Brant Houston. 2017. A breed apart? comparative study of investigative journalists and US journalists. Journalism Studies, 18()):265-287.
Cathy O'Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown Publishing Group, USA.
Gordon Pennycook, Adam Bear, Evan T. Collins, and David G. Rand. 2020. The Implied Truth Effect: At- taching Warnings to a Subset of Fake News Headlines Increases Perceived Accuracy of Headlines Without Warnings. Management Science, 66(11):4944 4957.
David M. J. Lazer, Matthew A. Baum, Yochai Ben- kler, Adam J. Berinsky, Kelly M. Greenhill, Filippo Menczer, Miriam J. Metzger, Brendan Nyhan, Gor- don Pennycook, David Rothschild, Michael Schud- son, Steven A. Sloman, Cass R. Sunstein, Emily A. Thorson, Duncan J. Watts, and Jonathan Zit- train. 2018 The science of fake news. Science, 10946330))1944100990
Neil Perry. Megha Srivastava, Deepak Kumar, and Dan Boneh. 2022. Do users write more insecure code with ai assistants? arXiv preprint arXiv:2211.0362
Chris van der Lee, Albert Gatt, Emiel van Miltenburg. Sander Wubben, and Emiel Krahmer. 2019. Best practices for the human evaluation of automatically generated text. In Proceedings of the 12th Interna- tional Conference on Natural Language Generation, pages 355-368, Tokyo, Japan. Association for Com- putational Linguistics.
Pontus Plavén-Sigray, Granville James Matheson, Bjöm Christian Schiffler, and William Hedley Thompson. 2017. The readability of scientific texts is decreasing over time. Elife, 6:e27725.
Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. 2018. A stylometric inquiry into hyperpartisan and fake news. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers). pages 231-240, Mclbourne, Australia. Association for Computational Linguistics.
Kobi Leins, Jey Han Lau, and Timothy Baldwin. 2020. Give me convenience and give her death: Who should decide what uses of NLP are appropriate, and on what basis? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2908-2913, 13, Online. Association for Computational Linguistics.
Manoel Horta Ribeiro, Shagun Jhaver, Savvas Zannet- tou, Jeremy Blackburn, Gianluca Stringhini, Emil- iano De Cristofaro, and Robert West. 2021. Do plat- form migrations compromise content moderation? evidence from r'the donald and r/inclls. Proc. ACM Hum. Hum.-Compee Comput. Interuct., 5(CSCW2).
Stephan Lewandowsky, John Cook, and Doug Lom- bardi. 2020. Debunking Handbook 2020. Accessed December 8th, 2022.

---

Luca Righes, Mohammed Saeed, Gianluca Demartini, and Paolo Papotti. 2023. The community notes obser- vatory: Can crowdsourced fact-checking be trusted practice? In Companion Proceedings of the ACM Web Conference 2023, WWW '23 Companion, page 172-175, New York, NY, USA. Association for Com- puting Machinery.
Chenhao Tan, Vlad Niculae, Cristian Danescu- Niculescu-Mine and Lillian Lee. 2016. Winning ar- guments: Interaction dynamics and persuasion strate- gies good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web, WWW 16, page 613-624, Republic and Can- ton of Geneva, CHE. International World Wide Web Conferences Steering Committee.
Anna Rogers and Isabelle Augenstein. 2020. What can we do to improve peer review in NLP? In Find- ings of the Association for Computational Linguisties: EMNLP 2020. pages 1256-1262, Online. Association for Computational Linguistics.
Kai Xiang Teo. 2021. Civil Society Responses to Sin- gapore's Online "Fake News" Law. International Journal of Communication, 15:4795-4815.
The Royal Society. 2022. The online information en-
Anna Rogers, Timothy Baldwin, and Kobi Leins. 2021. 'jsst what do you think you'le re doing, dave?'a a check- list for responsible data use in nlp. Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4821-4833.
vironment: Understanding how the internet shapes people's 's engagement with scientific information.
James Thome and Andreas Vlachos. 2018. Automated fact checking: Task formulations, methods and fu- ture directions. Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 3346-3359, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
Mohammed Saeed, Nicolas Traub, Maelle Nicolas, Gi- anluca Demartini, and Paolo Papotti. 2022. Crowd- sourced fact-checking at twitter: How does the crowd compare with experts? In Proceedings of the 31st ACM International Conference on Informa- tion & Knowledge Management, CIKM '22, page 1736-1746, New York, NY, USA. Association for Computing Machinery.
James Thorne, Andreas Vlachos, Christos Christoddoulopoulls,, and Arpit Mittal. 2018. FEVER: large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies. Volume (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguisties.
Zeve Sanderson, Megan A. Brown, Richard Bonneau, Jonathan Nagler, and Joshua A. Tucker. 2021. Twitter flagged Donald Trump's tweets with election misin- formation: They continued to spread both on and off the platform. Harvard Kennedy School Misinforma- fion Review.
Fabio Tollon. 2022. Artifacts and affordances: from designed properties to possibilities for action. Al & SOCIETY, (7(1)2239-248.
Hendrik Schuff, Alon Jacovi, Heike Adel, Yoav Gold- berg, and Ngoe Thang Vu. 2022. Human interpreta- tion of saliency-baa based explanation over text. FAceT '22, page 61 1-636, New York, NY, USA. Association for Computing Machinery.
Rudra M. Tripathy, Amitabha Bagchi, and Sameep Mehta. 2010. A study of rumor control strategies on social networks. In Proceedings of the 19th ACM international conference on Information and knowl- edge management- CIKM '10, page 1817, Toronto, ON, Canada. ACM Press.
Tal Schuster, Roci Schuster. Darsh Shah, and Regina Barzilay. 2020. The limitations of stylometry for detecting machine-generated fake news. Computa- tional Linguisties. 46(2):499-5 510.
Ibo van de Poel. 2020. Embedding values in artifi- cial intelligence (ai) systems. Minds and Machines, 30(3):385-403
Norbert Schwarz and Madeline Jalbert. 2021. When (fake) news feels true: Intuitions of truth and the acceptance and correction of misinformation.
Helena Vasconcelos, Matthew Jörke, Madeleine Grunde- McLaughlin, Tobias Gerstenberg, Michael S Bern- stein, and Ranjay Krishna. 2023. Explanations can reduce overreliance on ai systems during decision- making. Proceedings of the ACM on Human- Computer Interaction, (CCSCWI)11-38..
Craig L. Silverman. 2013. Verification handbook an ultimate guideline on digital age sourcing for emer- gency coverage.
Thomas Simpson. 2013. Evaluating google as an epistemic tool. Philosophical Engineering: Toward a Philosophy of the Web, pages 97-115.
Andreas Vlachos and Sehastian Riedel. 2014. Fact Checking: Task defimition and dataset construction. In Proceedings of the ACL 2014 Workshop on Lan- guage Technologies and Computational Social Sci- ence, pages 18-22, Baltimore, MD, USA. Associa- tion for Computational Linguistics.
Mark Spilsbury. 2017. Diversity in journalism. Na- tional Council for the Training of Journalist UK: Essex, UK, pages 4-10.
Baruch Spinoza. 1660. On the uses of the foregoing. In Short Treatise on God, Man and His Wellbeing, chapter 18.
Emily K. Vraga, Sojung Claire Kim, John Cook, and Leticia Bode. 2020. Testing the Effectiveness of Correction Placement and Type on Instagram. The

---

International Journal Press/Politics. 25(4):632-
652.
1,000
Nathan Walter and Riva Tukachinsky. 2020. Meta- Analytic Examination of the Continued Influence of Misinformation in the Face of Correction: How Powerful Is It, Why Does It Happen, and How to Stop It? Communication Research, 155-11757777
Citations
500
#
William Yang Wang. 2017. "liar, liar pants fire": new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( (Volume Short Papers), pages 422 -226.
0
0
20
40
60
80
100
Rank
Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Per- suasion for good: Towards personalized persuasive dialogue system for social good. In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 5635-5649, Florence, Italy, Association for Computational Linguisties.
Figure Number of citations per paper in our set, or- dered according to rank.
Appendix
Yaqing Wang. Fenglong Ma, Zhiwei Jin, Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. 2018. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 849 857. ACM.
A Providing veracity labels
Following our findings with respect to unsupported ends (see Section 6.3), we analysed the evolution over time of a particular case: providing verac- ity labels or scores to human fact-checkers as a means of limiting misinformation. Human fact- checkers have expressed strong skepticism the usefulness of such scores for their work (Nakov al., 2021). Thankfully, providing veracity labels to users general is a declining trend. Organiz- ing our data temporally (see Figure 7), we find that the percentage of papers proposing that users should act directly on model veracity prediction has decreased, whereas the percentage of papers proposing to identify potential claims for humans to fact-check, provide evidence for humans to act on, actually replace human fact-checkers (i.e.. write full articles discussing the evidence for and against claims) has increased.
Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, 7ac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and lason Gabriel. 2022. Taxonomy of risks posed by language models. In 2022 ACM Conference Fairness, Accountabil- ity, and Transparency, FAceT '22, page 214-229, New York, NY, USA. Association for Computing Machinery.
Takuma Yoneda, Jeff Mitchell, Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. UCL ma- chine reading group: Four factor framework for fact finding (HexaF). In Proceedings of the First Work- shop on Fact Extraction and VERification (FEVER), pages 97-102, Brussels, Belgium. Association for Computational Linguistics.
B Annotation Guidelines
To study the epistemology of automated fact- checking ("AFC"), we annotate the narratives 100 highly cited research papers.
Kevin S Zollman. 2007. The Communication Struc- ture of Epistemic Communities. Philosophy of Sci- ence, 74(5):574- 587.
We define a two-step annotation scheme: 1) a paragraph-level annotation and 2) discourse-level narrative annotation. In the paragraph-level annota- tion, extract quotes related to the goal and the methodology presented by identifying identify the a) data subjects, b) actors, c) model means, d) appli- cation means, and e) epistemic ends. Then, based on the identified elements, we extract the implied narratives in the discourse- annotation.

---

Year
2009
2014
2015
2016
2017
2018
2019
2020
2021
Papers
4
5
11
23
15
28
11
Table 1: Papers per year in the sample of 100 highly cited papers we worked from.
80
60
Frequency of narrative (%)
Frequency of means (%)
60
11
20
20
0
2016
2018
2020
2016
2018
2020
Year
Year
Figure 6: Evolution over time for providing veracity la- bels, other assistive means, and full automation. We see steady decline in the percentage of papers expecting to directly present veracity labels to data actors. Note that the numbers add 100%, as our annotation is multi-label.
Figure 7: Evolution over time for vague identification, vague opposition, and vague debunking.
extract quotes
B.2 Paragraph-Level Annotation
We list the long-form definitions of epistemic ele- ments we developed for consistent annotation here, along with the percentage of papers found each element in. We include also category labels pected based on our initial discussions, but which did not appear our sample.
We went through three-round annotation process. First, annotated small pilot set of five papers define an annotation scheme. Then, working inductively, we annotated the full of (100) re- search papers. That is, if, during the annotation, we encounter a means, end, actor, or narrative that does not fit any of our given categories, we intro- duce a new one. We then discussed our annotations and unified our set of introduced categories into those discussed in Appendixes B.2 and B.3. We further created flowchart help us move from paragraph-level discourse-level annotation in structured way (see Appendix B.5). Finally, we re-annotated all 100 papers based on our unified set categories. The final set of criteria can be found in the sections below.
B.2.1 Quotes
We extract the quotes/paragraphs from the intro- duction, yet, if a piece of information is missing, we further look at the abstract. We only examine epistemic quotes, i.c., related to knowledge. We sort them into quotes answering questions about narratives, i.e., the why and what of the paper.
B.2.2 Data subjects
Based on "who did what to whom for whom", ", we extract the subjects/pcople whose texts are fact- checked. They can be:
B.1 Paper Selection
We collect research papers on automated fact- checking and related tasks (e.g., rumor detec- tion). Working from the GitHub repository of fact-checking papers published alongside Guo et al. (2022)3, we select the 100 most cited. We exam- ine the introductions and abstracts of each paper to
journalists,
citizen journalists,
social media users,
technical writers,
Chhck our GitHub repository for links to the papers.
public figures/politicians,

---

product reviewers.
media consumers,
Social media users (24.9%) Covers any (poten- tially anonymous) contributors to social media as long as they are not explicitly public figures. This category includes people who comment on forum, TwitterFacebook users, or editors of Wikipedia and other collaborative writing projects.
technical writers,
engineers and curators.
law enforcement,
algorithm.
Professional Journalists (15%) Covers profes- sional journalists, including fact-checkers, and any- one writing at a publishing house. This category also includes online publishers, but not collectives of amatcur sleuths, c.g., Bellingcat. Organizations pretending to be journalists, e.g., satire sites and fake news websites, are also counted within this category.
Professional Journalists (18.5%) Covers profes- sional journalists, including fact-checkers, and any- thing written at publishing house. This category does not include amateur sleuths, c.g., Bellingeat, or bloggers doing the work of journalists.
Citizen Journalists (2.6%) Covers amateur jour- nalists who take on the same work professionals without funding traditional education, c.g.. blog- gers, social media users, and collectives such as Bellingcat.
Citizen Journalists (0.4%) Covers amateur jour- nalists who take on the same work as professionals without funding or traditional education, c.g., blog- gers, social media users, and collectives such as Bellingeat.
Social Media Moderators (0%) Covers people hired to moderate social media spaces. Applica- ble only when it is explicitly stated that a human employee should act on the model outputs.
Politicians & Public Figures (5.6%) Covers any public figure, such as politician or an actor. Analysed data could be media releases, interviews, speeches, or similar.
Covers scientists, as well as any other actors who would use model outputs for scientific research. E.g., to analyse data with the express purpose of learning something about not acting on the model decisions.
Scientists (3.9 %
Product Reviewers (1.7%) Covers specifically product reviewers sites such Amazon, Trust- pilot, where the purpose of the commenter to describe and rate a product.
Media Consumers (4.7%) Covers ordinary peo- plc who consume the content to which the fact- checking system is supposed to be applied. Only applicable when the consumer is directly under- stood (possibly implicitly) to use the system, c.g., in the form of a browser extension. The decision to use the outputs of the tool must be in the hands of the consumer.
Technical Writers (1.7%) A few papers suggest scientists and other writers of technical documents should use fact-checking e.g., spot errors in their articles before publication. Along with scientific writers, this also covers writers of technical docu- ments for areas such as business or health, as well as lawyers and clerks seeking to ensure consistency within legal documents.
Engineers and Curators (1.3%) Covers cases, where engineers curators are maintaining a knowledge base of some kind, (e.g., Wikipedia), are intended use the model outputs in their work.
B.2.3 Data actors
The people who are supposed to act on the model outputs, such as journalists, social media modera- tors, media users. From our data, they can be
Law Enforcement (0.9% Covers cases where law enforcement agents, e.g., police officers, intel- ligence agents, or judges, are intended act on the model outputs.
professional journalists,
citizen journalists,
Algorithm (1.7%) Covers the cases of fully au- tomated systems that act on the model outputs, e.g., remove posts based on the model's predictions.
social media moderators,
scientists,

---

B.2.4 Model owners
Classify/score Veracity (79.3%) When the au- thors propose classify the veracity of claims, i.c., whether or not the claim is true (or supported by evidence).
The entities who own the models, or institutions who employ those who act on the model outputs. For instance, fully automated moderation systems are owned by the companies expected to use the models, c.g., social media companies. From our data, they can :
Classify/score Stance (3.9%) When the authors propose to classify the stance of evidence, i.c.. whether not evidence document takes a pos- itive or negative view on particular subject claim.
media companies,
social media companies,
Evidence Retrieval (24.5%) When the authors propose evidence retrieval as a mean to reach their goal.
law enforcement.
Media companies (0.4%) Covers professional (non-amateur) media companies employing jour- nalists and fact-checkers.
Produce Justifications (6.9% When the authors propose generating explanations/jsstiicattosss reach their goal.
Social Media Companies (4.7%) Covers social media companies more generally, including the engineers working to maintain the social network. Applicable when decisions will be made automati- cally based on the model's decisions, when it is unclear.
Human in the loop (11.2%) When there are hu-
man actors involved in the solution/main process described in the paper.
Corpora Analysis (2.1%) %)) When the authors pro- pose using data analytics or any sort of corpora analysis in their methodology.
Law Enforcement (0.9%) Covers cases where law enforcement agents, e.g., police officers, intel- ligence agents, or judges, are intended act on the model outputs.
Generate claims (0.9%) When the authors pro- pose generating claims, e.g., produce additional misinformation to train on.
B.2.6 Application Means
Scientists (1.3%) Covers scientists, well as any other actors who would use model outputs for scientific research. F.g., analyse data with the express purpose of learning something about it, not acting on the model decisions.
How do the authors want to use what developed in the paper to accomplish a specific goal (e.g., re- duce the spread misinformation)? For example, this could be by deploying automated systems to show social media users content warnings about claims which might be false (along with, poten- tially, evidence for their falsity). From our data, we identify the following suggestions:
B.2.5 Modeling (ML) Means
What concretely do the authors propose to do in terms of machine learning models? E.g., classify- ing claims, finding evidence, or similar. From our data, these can be:
identify claims,
triage claims,
classify/score veracity,
supplant human fact-checkers,
classify/score stance,
gather and present evidence,
evidence retrieval,
identify multimodal inconsistencies,
produce justifications,
automated removal,
corpora analysis,
provide veracity labels/scores,
human in the loop.
provide aggregates of social media comments,
generate claims.
filter system outputs,

---

maintain consistency with KB,
Filter system outputs (3.0%) Other NLP/ML models, e.g. LLMs, struggle produce truthful outputs. Fact-checking models should filter or re- rank their output. This also includes extractive models, e.g., relation extraction systems, that are coupled with fact-checker to only return (or add to KB) things the fact-checker accepts as truthful.
analyse data,
produce misinformation,
vague persuasion.
Maintain consistency with KB (1.3%) Internal knowledge bases (defined loosely, including textual ones, such as Wikipedia) can be used source of truth to prevent the data actor from publishing untruths. This could be at writing time, e.g., as a writing assistant, or it could be by continuously keeping an article published online to date.
Identify claims (26.2%) There are many claims on the internet and most of them are misinfor- mative. ML should be deployed to find the misin- formative ones.
Triage claims (6.0%) Fact-checkers, content moderators, and similar have many claims deal with. ML should be deployed rank them, so more costly actors focus on the most important ones first.
Analyse data (3.0% 3.0%) Fact-checking models and datasets should be used for research purposes, e.g., to analyse misinformative text to get better under- standing of how misinformation spreads.
Supplant Human fact-checkers (13.7%) Re- place human fact-checkers entirely, least partially by automatically handling some claims (in fully automated fashion). If the intention is a human-in -nttan--itthh-ooop system, supplanting human fact-checkers not the means.
Produce misinformation (0.9% machine learning model that produces misinformation can be used against misinformation, for example, to generate adversarial data, or to show people what such models are capable of to innoculate them against it when they encounter it in the wild.
Gather and present evidence (10.7%) An ML model should find relevant evidence support- ing/refuting a claim, and show it to a human. It can also involve generating justifications for how the evidence relates to the claim.
Vague persuasion (3.4%) ML models should somehow convince people to change their opinions on claims, e.g., by providing warning labels or pre- senting evidence. The mechanism is not specified, though.
Identify multimodal inconsistencies (1.7%) For multimodal misinformation, an important tool identifies mismatches between modalities. ML models should do this, then show the results to humans.
B.2.7 Ends
The purpose or ultimate aim of the approach. What do the authors want to accomplish? From our data:
limit misinformation.
limit Al Al-generated misinformation.
Automated removal (1.7%) ML models should automatically remove claims from e.g., social me- dia platforms, with no human involvement.
increase the veracity of published content,
develop knowledge of NLP/language,
Provide veracity labels/scores (17.5%) MI_
avoid biases of human fact-checkers.
models should provide indications of truth value data actors, either in the form of labels veracity scores.
detect falsehood for law enforcement
Limit misinformation (78.1%) The ultimate aim of the paper is to prevent misinformation from spreading to limit its influence.
Present aggregates of social media com- ments (6.9%) ML models should aggregate the stance/evidence presented in the comments potentially misinformative claim, as a way of sum- marizing points made by humans in favor of or against the claim.
Limit Al-generated misinformation (3.4% The ultimate aim of the paper is to prevent AI- generated misinformation from spreading or to limit its influence.

---

Increase veracity of published content (7.2%) Some systems are intended be used by publish- ers and writers before their content is made public -or applied to continuously keep published content up to date. The aim here not to limit already spreading misinformation but to keep people from accidentally publishing untrue things. Published content can be small data, like single article, large collection of data, like Wikipedia, knowl- edge base.
assisted knowledge curation,
scientific curiosity.
truth-telling for law enforcement
vague moderation,
adversarial research.
We do not account for implied paragraph-Ievel narratives but only for annotated elements that *are* in the quotes. As we extract multiple quotes, papers may have more than one narrative present.
Develop knowledge of NLP/languae (3.4%)
Automated fact-checking is difficult problem. By studying it, we can learn more about how language works, and how to build models that interpret se- mantics.
Vague Identification (31 %) The paper mentions identifying or detecting misinformative claims the means, and limiting misinformation as the end. However, it is not clear how the authors intend accomplish that end using those means. Typically. there are no data actors - it is also not clear who should act on the model's predictions.
Avoid biases of human fact-checkers (0.4% I.e., develop "super-human" fact-checking.
Detect falsehood for law enforcement (0.9%)
Vague identification Applies only in cases where the authors say that they want to identify detect misinformation without saying what they are going to do with that afterward, e.g., in rumor detection papers where they claim that they want detect rumors but we do not know what they want to do with this identification or classification labels. A fictional example follows below:
One proposed use case for fact-checking and simi- lar technologies is truth-telling systems for law enforcement, e.g. in courtrooms.
B.2.8 Important Note
this annotation stage we do extract implied statements, we rely directly on extracted quotes and do not interpret them.
"Misinformation is major societal problem, eroding community trust and costing lives by e.g. inducing hesitance adopt life-saving vaccines. The amount of messages spread social media has increased drastically in recent years. Therefore, is necessary to automatically identify potentially misinformative claims in order to address this prob- lem."
B.3 Narratives
At the discourse level, we extract the epistemic narratives present in the paper. We envision narra- tives as discourse structures combining the (mod- eling/application) means, ends, data actors, and data subjects extracted at the paragraph level. Note that use fictional examples below avoid bias- ing annotators' decisions specific papers. The narratives can be:
Vague Debunking (14%) The authors propose to produce evidence-based debunkings of text via automated means, but it is not clear whether the introduced model is an assistive tool for fact- checkers or fully automated. Further, it not clear how the "debunkings" will communicated to misinformation--elieeeee
vague identification,
vague debunking,
vague opposition,
assisted content moderation,
It is clear that the mechanism is supposed to follow what fact-checkers are currently doing, but not where or how the MI model will be used in this process. Furthermore, it is unclear whether the entire process will be automated. For example, the paper may suggest that an automated fact-checking model should be used by fact-checkers, but is not clear how the model assists in that. A fictional example follows below:
automatic content moderation,
assisted media consumption,
assisted internal fact-checking.
assisted external fact-checking.
automated external fact-checking,

---

Category Label
Description
Vague identification
Identify misinformation without explaining what happens afterwards.
31
Automated External Fact-checking
Fully automate the work of journaíistic fact-chockers. Opposition to misinformation with clear application means.
22
Vague opposition
19
Assisted External Fact-checking
Assist journalists writing fact-checking articles e.g., by finding evidence. Evidence-baad based debunking with no clear users.
18
Vague Debunking
14
Assisted media consumption
Assist media consumers e-g. providing extra information.
8
Scientific curiosity
Develop AFC artefacts to learn something about e.g., language NLP. Assist humans in curating public knowledge bases, c-g.. Wikipedia.
Assisted Knowledge Curation
Assisted internal fact-checking
Use AFC journalistic contexts fact-check before publication.
Automated content moderation enforcement
Fully automate content moderation social media platforms. Use AFC truth-telling system in law enforcement contexts.
Table 2: Epistemic narratives found in our analysis of 100 fact-checking papers. Along with each narrative, we list the percentage of papers in which we found it.
E.g.. "Misinformation a major societal prob- lem, eroding community trust and costing lives by e.g., inducing hesitance to adopt life-saving vac- cines. One proposed solution is to debunk circu- lating claims, i.e. to find evidence against them. This is the process commonly carried out by fact- checking organizations, e.g. PolitiFact. In this paper, we introduce a classifier..."
forms, the means is to provide suggestions for posts to delete (along with, potentially, evidence for why
they might be false), and the actors are human mod- erators who make the final choice on whether posts should be deleted not. A (fictional) example follows below:
"Social media is rife with misinformation, erod- ing community trust and costing lives by e.g., induc- ing hesitance to adopt life-saving vaccines. One solution is for moderators to remove information deemed false. However, with the number of posts made every day on social networks, this strategy is too costly. In this paper, we develop an auto- mated system for filtering claims, helping modera- tors quickly discover and make decisions on circu- lating claims."
Vague Opposition (19%) Restricted to cases without any application means (model means and application means in contrast to vague identi- fication and vague debunking). E.g., a machine learning/automated system will reduce the spread of misinformation.
When the paper presents a narrative of vague op- position to misinformation. The end is to limit the spread or influence of misinformation, and the ML means are, for example, to classify claims. How- ever, the connection between means and ends is left unmentioned, and epistemic actors are typically ab- sent. An impression is given that the development of automated fact-checking will limit the spread of misinformation, but the link between the two is left unstated. A (fictional) example follows below: "Misinformation is a major societal problem, eroding community trust and costing lives by e.g., inducing hesitance to adopt life-saving vaccines. It is therefore of paramount importance that the spread of false information is stopped. Automated fact-checking - that is, the automatic classification of claim veracity- - represents one solution to this critical problem."
Automated Content Moderation (4%) If the pa- per analysed proposes similar content moderation strategy, but instead of assisting human moderators, it suggests replacing them entirely. In this case, the end is to limit the spread of misinformation on social media platforms, the means to deploy classifiers to truth-tell claims and remove any la- beled false, and the actors are the algorithm (as well as, implicitly, the model owners - the execu- tives and engineers at social media companies who deploy and make decisions about such systems). A (fictional) example follows below:
"Social media is rife with misinformation, erod- ing community trust and costing lives by e.g. induc- ing hesitance to adopt life-saving vaccines. One solution is for moderators to remove information deemed false. However, with the number of posts made every day on social networks, this strategy is too costly. In this paper, we develop an automated system for detecting false claims, which can serve as a first line of defense against misinformation."
Assisted Content Moderation (0%) If the pa- per proposes the deployment of automated fact- checking as a tool to assist content moderators on social media platforms. Here, the end is to limit the spread of misinformation on social media plat-

---

Assisted Media Consumption (8%) the paper proposes to deploy automated fact-checking as an assistive tool for consumers of information, either as a layer adding extra information to social media posts or as a standalone site where claims can be tested. In this case, the end is either to limit the influence of misinformation or to induce veracious mental states in users; the means is to deploy auto- mated systems to warn about claims which might be false (along with, potentially, evidence for why their falsity); and the actors are social media users or information seekers in general. The actors could also be social media companies who show informa- tion produced by fact-checking assistants to users as an integral part of their UI. A (fictional) example follows below:
Assisted External Fact-checking (18%) When the paper proposes to deploy automated fact- checking as an assistive tool for journalists, de- ployed for external fact-checking. Here, the end is to limit the influence of misinformation, the means is either to speed up the production of counter- messaging by surfacing relevant evidence or to di- rect journalists to the most problematic currently circulating claims, and the actors are either journal- ists employed at fact-checking organizations such as Full Fact or citizen journalists.
This is restricted to when the paper proposes to improve one or multiple components in the auto- mated fact-checking pipeline rather than the whole pipeline/end-to-cit system (in the latter case it becomes automated external external fact-checking, see fol- lowing paragraph). Lee. when it is human-in-the- loop, then it is assisted fact-checking but not auto- mated.
"Misinformation is a major societal problem, eroding community trust and costing lives by e.g. inducing hesitance to adopt life-saving vaccines. It is therefore of paramount importance that the spread of false information stopped. Studies have shown that many people adopt beliefs without doing due diligence on the information they receive. Automated fact-checking that is, the automatic classification of claim veracity - could via e.g., a plugin be deployed to warn social media users about potentially false claims."
A (fictional) example follows below:
" "Misinformation is major societal problem, eroding communitty trust and costing lives by e.g., inducing hesitance to adopt life-saving vaccines. An important way to fight misinformation the production of relevant counter-mest meunter-mssaaging, i.e., the work done by organizations such as Full Fact or PolitiFact. With the number of false claims pub- lished on social media every hour, it is not fea- sible for human journalists to debunk them all. Journalists could use automated fact-checking triage incoming claims to limit the workload, or quickly surface relevant evidence while producing articles."
Assisted Internal Fact-checking (4%) When the paper proposes to deploy automated fact- checking as an assistive tool for journalists, de- ployed internally. Here, the end is to increase the veracity of published information, the means is to deploy automated systems to warn about claims which might be false (along with, potentially, evi- dence for why their falsity), and the actors are ei- ther journalists employed at traditional publishing houses or citizen journalists. A (fictional) example follows below:
Automated External Fact-checking (22%) When the paper proposes to fully automated exter- nal fact-checking, including all parts of the pipeline. end is to limit the influence of misinformation, the means are to entirely replace human fact-checkers by automatically producing fact-checking articles, and the actors are users engaging with the produced content. It is only automated external fact-checking when the authors explicitly say the process should be automated, otherwise it is vague debunking (see previous paragraph).
"Research is a fundamental iask in journalism, conducted to ensure published information is truth- ful and to protect the publisher from libel suits. This is a crucial step, which journalists - strained by the advent of the 24-hour news cycle increas- ingly skip. Given trusted source of evidence docu- ments, such as LexisNexis, much of the grunt work of research could be handled by automated fact- checkers, leaving journalists free to tackle the hard- est parts, e.g. Mouble-checkeeg information with sources."
A fictional example can be seen below:
"Misinformation is major societal problem, eroding community trust and costing lives by e.g.. inducing hesitance to adopt life-saving vaccines. An important way to fight misinformation the production of relevant counter counter- messaging, i.e., the work done by organizations such as Full Fact or

---

PolitiFact. With the number of false claims pub- lished on social media every hour, is not feasible for human journalists to debunk them all. As such, the process must be automated."
cessing and adjacent fields. (fictional) example follows below:
"Journalistic fact-checking is a difficult task, re- quiring reasoning about disputed claims that fool sufficiently many humans to warrant professional attention. For systems mimic fact-checking substantial degree, significant semantic under- standing necessary. As such, automated fact- checking is an ideally suited field to develop and test new models for natural language understand- ing."
Assisted Knowledge Curation (7%) When the paper proposes fact-checking primarily as a compo- nent filtering the information kept in some curated knowledge vault, including graph- knowl- edge bases as well as text-based collections such as Wikipedia. Here, the end is to increase the veracity of the knowledge vault, the means is to use auto- additional truth-telling
mated fact-checking layer that prevents disputed facts from being added (or interrogates already added facts), and the actors are typically the engineers who maintain the knowl- edge base. A (fictional) example follows below: "Knowledge bases fuel many real-world NLP applications. e.g., question answering. The mainte- nance of knowledge bases is an expensive process, yet new facts appear in the world knowledge bases must be kept up-to-date. Automated triple extraction from e.g., news data has been proposed as an alternative to human annotators; yet, the quality remains low. Automated fact-checking sys- tems, which verify facts against trusted knowledge sources, could be used to prevent highly disputed facts from being entered - or ensure that new facts are consistent with the existing knowledge."
B.4 Feasibility Support
We annotate narratives with any support for their feasibility given in the paper. Vague narratives are by definition not supported - if the narrative is clear enough to design e.g.. a user study to test if the means are an effective way to reach the end, it is not vague. We use the following categories:
Scientific Research The feasibility of the narra- tive is supported by reference to scientific studies. This only applies if the entire narrative is supported, i.e., a scientific paper is supported for the means being feasible way to reach the end. This Does NOT apply if only e.g., the dangers of misinforma- tion supported the proposed means being effective strategy MUST be entirely supported. For example, the crowdworkers' study in the FactCheckingBriefss org/ 2020 2wwp:mmm0pmppmpp-.mmmmmmmmmmmmmm maiiippmmiin...m 580/) could be cited to demonstrate that giving people evidence increases their accuracy veracity judgments. This applies even if the cited paper does not support what the paper claims it supports, e.g., someone cites the FEVER dataset (Thorne et al., 2018) for fully au- tomated fact-checking being an effective strategy to reduce the spread of real-world misinformation. This almost always applies to narratives of scien- tifec curiosity, as researchers tend to show their research is useful strategy for studying what they investigate by writing a related works section.
Truth-telling for Law Enforcement (1%) When the paper proposes fact-checking primarily as a lie detector for use in law enforcement con- texts. This could include police work as well as courtroom applications. A fictional example can be seen below:
"Automatic detection of deception is commonly used in police work via e.g. polygraphs. However, accuracy remains low. We propose that automated fact-checking via NLP could represent a viable alternative."
Scientific Curiosity (7%) When the authors of the paper justify their projects purely based on sci- entific curiosity. While differing strongly from the other narratives presented here, this is still a virtue epistemic narrative, concerned with the production of good knowledge. Hcre, the end is to increase scientific knowledge of semantics; the means is to learn how build automated systems that mimic human fact-checkers, process theorised yield knowledge about the construction of meaning; and the actors are scientists in natural language pro-
News Media The feasibility of the narrative is supported by reference to a newspaper article. This occurs in the same cases as scientific research, except the citation is to newspaper article rather than a scientific article.
The feasibility of the narrative is supported by reference to the studied task currently working as a human task. It is assumed that full
Automation

---

partial automation will work, and will be effec- tive i.c., no reference studies testing whether the proposed strategy helps humans in the loop given.
Example "While the number of organizations performing fact-checking is growing, these ef- forts cannot keep up with the puce at which false claims are being produced, including also clickbait (Karadzhov et al., 2017a), hoaxes (Rashkin al., 2017), and satire (Hardalov et al., 2016). Hence, there is need for automatic fact checking." " (Baly et al., 2018)
Vague Community The feasibility of the narra- tive is supported by reference to the preference of practitioners, usually without citing scientific pa- per. This includes phrases like "many people think this is good idea".
Example "Many favor a two-step approach where fake news items are detected and then coun- termeasures are implemented to foreclose rumors and to discourage repetition." (Potthast et al., 2018)
The feasibility of the narrative is not directly supported. However, the paper argues that the strategy represented by the narrative (e.g.. automated content moderation) must be done, or bad things will happen. The bad thing is something other than human workers not being able to keep up with their workload, the narrative otherwise falls under automation.
Sense of Threat
Example "An abundance of incorrect informa- tion can plant wrong beliefs in individual citizens and lead misinformed public, undermining the democratic process. In this context, technology to automate fact-checking and source verification (Vlachos and Riedel, 2014) is of great interest to both media consumers and publishers." (Yoneda et al., 2018)
B.5
Paper Metadata
The Type of the Paper Survey, dataset, model description, task description, other (name).
The Subfield/task Can be fact-checking, rumor detection, misinformation, disinformation, or other (name the task).
The Subtask
The part of the pipeline in which the model interacts, e.g., claim detection, evidence retrieval, verdict prediction, justification produc- tion.

---

Cara + powne
AOAEE noard
Aution
be
No
You
fravagge:
Tame Mercc
Yeu
Yea
Sedjec
NS
kog?
You
Autamiad eaheral tacicledingg
Automaand comte
Austrad techchesling
Asstiid conmumption
Ausited chesding
Figure 8: Flowchart of the Narrative Annotations.

---

Vagu
The profereroe soeee community
Newes Medta
new ancee
nathor masgpett this shoudd be usedd fedd The typothedd that m TIAuT 0re end!
that aggunentt supported by
Nosting
Yes
Invesggare
102
No
Does the poper directty argue thtt the menns are efochiv resthing
arpur an means meunssshoo should be 5u9 or partally automated Nwthout enteroa that
towes the papper arppan that tasd to ttrrpe meeen not well out wethout support from eviennoy)
No
No
Y00
Yee
Narroive
Arrmatton
Seevee
Figure Flowchart of the Feasability Support Annotations.


Reviews:
[{"review_id": "kZtCRKktuR", "reviewer": "Reviewer_21ex", "summary": "", "questions": "\n- Are there scoring frameworks that could be designed to assign a numerical value to find potential candidates for fact-checking?\n\n- What are good datasets for training and testing that might be assembled, either real or synthetic? (Generating synthetic datasets should be an interesting problem since the proposed framework could be inverted) Could some of these be done without doing deep NLP? (for instance, by doing purely propositional theorem proving, or by solving for evidence graphs, etc)\n\n", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments. We are happy they found our paper interesting and timely, despite the low score.\n\n**Appropriateness for EMNLP:**\n\nWe disagree that this paper is inappropriate for EMNLP. The conference explicitly invites surveys and position papers, as stated in the call for papers https://2023.emnlp.org/calls/main_conference_papers/. Moreover, this paper is a submission to the Ethics in NLP track, and EMNLP in general welcomes papers on ethics. \n\nWe further want to clarify that our paper and guidelines are primarily intended for *authors* of automated fact-checking papers and related tasks (e.g., rumour and deception detection), although of course, we believe that readers and reviewers are likely to benefit as well. Many of the papers we analyse were published at EMNLP, including some of the most cited (for example, [1]) and several of the papers we analyse were produced in connection to the FEVER workshop which took place three times at EMNLP.\n\n[1] Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking. Rashkin et al., 2017.\n\n**Automating Annotation:**\n\nIf we understand the reviewer correctly when they ask for *“an automated scoring algorithm for fact-checkers -- taking as input algorithms and datasets, and outputting a completeness and correctness score for each”*, the question is whether our content analysis process for papers could be automated. We believe automated content analysis is a very interesting direction. We would like to see how our analysis is received by the community first, to judge the usefulness of datasets and models for this; if the reception is positive, we would be happy to work towards this. The annotations we conducted will be useful to anyone who wishes to develop this kind of automated analysis.\n\n**Vague Narratives & Citations:**\n\nThe reviewer highlights that *“while the authors point out that evidence is often lacking for claims being made in narratives (and missing entirely for 'vague narratives'), they also acknowledge that even though citations are included, they need not always be relevant.”* What we mean in our paper is that empirical evidence for claims in papers is lacking for two reasons: 1) because no citations are mentioned, or 2) sometimes when citations ARE mentioned, they do not support the argument being made (i.e., they are “irrelevant” to the claim).\n\n**Claim Detection Scoring Frameworks:**\n\nSystems for finding potential claims to fact-check are an active area of research. Claim detection has also been the subject of several shared tasks, including RumourEval and FACTIFY. These are included in our analysis, but we will deepen the discussion of these in the camera-ready if the paper is accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is a discussion of a proposed structure for analyzing fact-checker algorithms, along with an overview of how this framework may be used to understand some of the more highly cited recent papers proposing solutions; as such, this is both interesting and important. Especially as we enter a time of increasingly easy generation of realistic looking narratives using AI, fact-checking, whether automated or not, will become increasingly critical. Having a consistent framework and baseline will make it easier to compare these systems against one another. As a starting point, future work in this area can extend the framework in interesting ways, stimulate discussions on how some of these metrics may be automated and how to compare or weight one dimension over another.\n\n", "reasons_to_reject": "I was hoping that this would be an automated scoring algorithm for fact-checkers -- taking as input algorithms and datasets, and outputting a completeness and correctness score for each (essentially an equivalent of Precision and Recall in information retrieval). Perhaps even a machine learning based analysis of language and communication cues that may, in conjunction with background information, may be used to find potential candidates for further checking. \n\nInstead, this is more of a thoughtful discussion or guidelines for how reviewers (or professional fact-checkers) might want to think about reading papers while evaluating them for their claims on fact-checking. The analysis of the 100 papers that the paper presents is somewhat underspecified, leading to ambiguity: for instance, while the authors point out that evidence is often lacking for claims being made in narratives (and missing entirely for 'vague narratives'), they also acknowledge that even though citations are included, they need not always be relevant. Checking a fact, even if there is a citation, requires chasing down the references, understanding them, and making a determination about the validity of the current 'fact'. The paper points out that citations/evidence in itself is not a silver bullet, since there are a number of ways in which the citations or evidence presented can be cause problems for the reader.\n\nThe paper is an interesting and very timely reminder of the problems that we should be thinking about, but EMNLP is not (in my view) the best forum for this work, given the focus of this work. An ethics or social science conference would be a better venue.   ", "questions_to_authors": "\n- Are there scoring frameworks that could be designed to assign a numerical value to find potential candidates for fact-checking?\n\n- What are good datasets for training and testing that might be assembled, either real or synthetic? (Generating synthetic datasets should be an interesting problem since the proposed framework could be inverted) Could some of these be done without doing deep NLP? (for instance, by doing purely propositional theorem proving, or by solving for evidence graphs, etc)\n\n", "paper_topic_and_main_contributions": "The paper is an analysis of 100 \"highly cited\" papers on the topic of automated fact-finding regarding the questions of (i) the motivation for the paper? Why was the solution being discussed? (ii) the means or mechanisms for doing this; the 'how', and (iii) who this proposed solution is intended for -- the participants in the scenarios being looked at.\n\nThe analysis was conducted by two annotators (presumably the authors) manually going through the abstracts and introductory sections of these 100 papers and marking passages/text-spans with a series of tags based on a set of guidelines that the paper proposes. The paper presents some of the statistics from this analysis.\n\n", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "W8FuydY6jk", "reviewer": "Reviewer_gXnU", "summary": "", "questions": "The numbers in \"()\" refer to the page numbers in the paper.\n\n(1)\nWhere does the term \"artifacts\" come from?\n\n(3)\nPut the epistemic elements and narrative types in a table along with short descriptions and simple examples. This can also be part of the appendix if too large for the body.\n\n(3)\nThere are many percentages given for epistemic elements and the narrative types, and many of them do not make sense to me. E.g., the given categories for \"Model Owners\" total in 8% - what about the other 92%? And the categories for \"Modeling means\" exceed 100% - how can that be? And the \"Vague narrative\" types should make up 56% (as stated in the paper (on page 4), but the percentages add up to 64%? There are more examples like this in the paper - make sure to correct them, or to clarify how the percentages relate.\n\n(4)\nWhat's the discourse level?\n\n(4) \nWhat is the difference between vague debunking and vague opposition?\n\n(6) \nChapter \"Evidence is not a silver bullet\" is well researched and the arguments made in it are all valid. However, the paper is about improving the usefulness of fact-checking related research, and stating these general psychological phenomena which cannot be prevented by fact-checking systems is misplaced here, in my opinion. You should at least give suggestions on how researchers can consider these points in their work.\n\n(7)\nOne point in your recommendations is to include the data actor in fact-checking papers. It totally makes sense to think from the perspective of the target group when developing a tool, but do you have any suggestions of how to involve them? Are you suggesting to actually invite a sample from the target group as evaluators? How are you going to know what they need if they have a special role which you are not familiar with?\n\n(15)\n\"Social Media Moderators (0%)\" ??", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive comments. We are thrilled they find the paper well-written, thorough, and valid. We hope the reviewer is right that the paper will influence authors to give stronger consideration to the practical and societal impacts of their technological artefacts!\n\n**Terminology – Definitions & Examples**\n\nWe thank the reviewer for highlighting their clarity concerns. We would appreciate it if the reviewer could specify which terms they find confusingly worded so that we can improve our definitions.\n\nWe do include simple definitions for each narrative and epistemic element, albeit in Appendices B.2 and B.3 – as the reviewer points out, this is a long list and as such was moved from the main body of the paper. We also include an example for each narrative in Appendix B.3, as well as a table like the one requested by the reviewer (Table 1). If accepted, in the camera-ready we will move Table 1 to the main text, and add an example for each epistemic element to the appendix.\n\n**Intuitive Findings**\n\nWe agree that our findings and recommendations seem quite intuitive. However, we do not know of any previous paper that studies this question or provides recommendations similar to ours. Documenting that the phenomenon exists and making the community aware of the problem is (hopefully) the first step to fixing it.\n\n**Follow-up Research**\n\nWe agree that answering why so much vagueness occurs is an interesting question. We are happy to theorise, and will include a section on this in the camera ready if our paper is accepted; perhaps authors simply prefer leaderboarding to evaluation that includes data actors? Perhaps they believe questions about effectiveness to be obvious – “of course automatically identifying potential misinformation will help us fight it”? However, without e.g., interviewing authors, we will have to hedge this as we do not have access to their internal thought processes as they are writing their papers.\n\nWe do believe this is an interesting direction for future research. Producing a scientific answer (rather than just a guess) would include for example ethnographic research with authors. [2] carried out such analysis with high-energy physicists and molecular biologists – we would find a similar study with automated fact-checking authors (or NLPers in general) utterly fascinating.\n\n[2] Epistemic Cultures: How the Sciences Make Knowledge. Knorr-Cetina, 1999.\n\n**Questions**\n\n_(1) Where does the term \"artifacts\" come from?_\n\n“Artefacts” is a common term in philosophy referring to deliberate constructs, originating from Aristotle – see https://plato.stanford.edu/entries/artifact. With that said, we actually picked up the term from the ARR Responsible NLP Checklist (https://aclrollingreview.org/responsibleNLPresearch/).\n\n_(3) Put the epistemic elements and narrative types in a table..._\n\nSee section on terminology above.\n\n_(3) There are many percentages given for epistemic elements..._\n\nOur annotation scheme is multi-label, as stated in lines 158-159: “This annotation is multi-label, i.e., for each element, one quote may have several (or no) labels.” This means percentages do not sum to 1. For example, a paper may include both “classify/score veracity” and “evidence retrieval” as modelling means, if e.g. the system introduced in the paper is a pipeline. We will make this clearer in the camera-ready if our paper is accepted.\n\n_(4) What's the discourse level?_\n\nDefined on line 177: “Annotation at the discourse level is annotation spanning multiple quotes.” For example, the narrative shown in Figure 1 includes an epistemic end from one quote, and a modelling means from another. This means the annotation is at the level of “discourse” (as opposed to paragraph/quote).\n\n_(4) What is the difference between vague debunking and vague opposition?_\n\nVague opposition: No discussion in the paper of how the developed artefacts will be used to fight misinformation (although it is claimed that fighting misinformation is the goal). Typically: no application means.\n\nVague debunking: The paper claims the developed artefacts will assist in the production of evidence-based refutations, and that this will fight misinformation. However, there is a missing link between producing debunkings and using them. Typically: no data actors. \n\nSee also lines 271-286, as well as the flowchart in Figure 7 on p23.\n\n_(6) Chapter \"Evidence is not a silver bullet\" is well researched..._\n\nThe “evidence is not a silver bullet”-section was intended to give a recommendation to authors, albeit implicitly. As we mention, the common solution given in papers to counteract the effect of bias in models is to rely on evidence. We argue that evidence does not present a complete solution (i.e., not a silver bullet) for this problem. As such, authors should be careful to make such claims. We will make this clearer in the camera-ready if our paper is accepted.\n\n_(7) One point in your recommendations is to include the data actor..._\n\nWe are indeed suggesting that evaluating based on the preferences and needs of the target user group (i.e. fact-checkers, media consumers, …) might be preferable to leaderboarding. This could include explicit involvement of people from that group, yes. We understand that this can be difficult in practice. We will develop this further in the camera-ready if accepted. Ideally, if authors intend claims like \"our system can help fight misinformation\" to be scientific, they should have empirical evidence for those claims. This could involve:\n\n* Testing whether the system performs as claimed in practice, using a sample from the target group (i.e., as done in [3] with crowdworkers).\n* Citing a paper that does involve such an analysis.\n* Relying on the expressed needs of people in those groups, e.g., [4], [5].\n\n[3] Generating Fact Checking Briefs. Fan et al., 2020. \n\n[4] Automated Fact-Checking for Assisting Human Fact-Checkers. Nakov et al., 2021.\n\n[5] The Challenges of Online Fact Checking. Arnold, 2020.\n\n_(15) \"Social Media Moderators (0%)\" ??_\n\nWe assume this question refers to lines 1269-1272 in the appendix, where we go through all the possible data actors. The list in the appendix includes our initial guesses as to what categories might be useful (see discussion of our annotation process on lines 146-148). We expected social media moderators to appear somewhere as data actors, but it didn't happen in any of the 100 papers we analysed. This is indeed quite surprising, especially as similar systems are in practice used for content moderation. This absence highlights why considering our recommendation to make the data actors explicit is important (as some authors may have implicitly considered them to be data actors).\n\n**Style and Presentation**\n\nThanks for the pointers, very helpful!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well written and structured. The research was conducted systematically and logically, and the arguments made are thorough and valid. When accepted, the paper will draw researchers' attention towards the usability of fact-checking systems. It might especially influence those who \"only\" develop a technological solution without considering its practical and societal benefit. The paper would also introduce a new paradigm for evaluating the usefulness of fact-checking papers and systems. It would raise awareness of important aspects to consider for all those who work on fact-checking systems and care about its value for the community.", "reasons_to_reject": "There are many categories and new terms introduced which are hard to distinguish without concrete examples. The wording of those terms is also somewhat confusing, so that it is hard to read the paper at the first time. Especially when there are terms with subtle differences, like many of the sub-categories for epistemic elements and narrative types in the paper, combining a few of the terms would have been a good idea in order to make those rememberable for the reader.\nOne might argue that many of the analysis results and recommendations are quite simple and intuitive. It might have been more useful to find out why the analyzed papers have those shortcomings of usefulness instead of supporting the fact that they do have them. In my opinion the authors should have addressed the \"why\" and at least come up with ideas for follow-up research.", "questions_to_authors": "The numbers in \"()\" refer to the page numbers in the paper.\n\n(1)\nWhere does the term \"artifacts\" come from?\n\n(3)\nPut the epistemic elements and narrative types in a table along with short descriptions and simple examples. This can also be part of the appendix if too large for the body.\n\n(3)\nThere are many percentages given for epistemic elements and the narrative types, and many of them do not make sense to me. E.g., the given categories for \"Model Owners\" total in 8% - what about the other 92%? And the categories for \"Modeling means\" exceed 100% - how can that be? And the \"Vague narrative\" types should make up 56% (as stated in the paper (on page 4), but the percentages add up to 64%? There are more examples like this in the paper - make sure to correct them, or to clarify how the percentages relate.\n\n(4)\nWhat's the discourse level?\n\n(4) \nWhat is the difference between vague debunking and vague opposition?\n\n(6) \nChapter \"Evidence is not a silver bullet\" is well researched and the arguments made in it are all valid. However, the paper is about improving the usefulness of fact-checking related research, and stating these general psychological phenomena which cannot be prevented by fact-checking systems is misplaced here, in my opinion. You should at least give suggestions on how researchers can consider these points in their work.\n\n(7)\nOne point in your recommendations is to include the data actor in fact-checking papers. It totally makes sense to think from the perspective of the target group when developing a tool, but do you have any suggestions of how to involve them? Are you suggesting to actually invite a sample from the target group as evaluators? How are you going to know what they need if they have a special role which you are not familiar with?\n\n(15)\n\"Social Media Moderators (0%)\" ??", "paper_topic_and_main_contributions": "The authors investigate the usefulness of research papers about automated fact-checking. They argue that their usefulness decreases without considering who will be using the developed fact-checking artifacts, and without specifying how they should be used. Therefore, they identify relevant components of a fact-checking system and document their degree of consideration in 100 popular fact-checking papers. \nThey contribute a holistic perspective on fact-checking systems, critically analyze a high number of existing papers, define influential components for the usefulness of fact-checking systems, and derive recommendations for future research. They document the results of the analyzed papers in the form of annotations, which is provided in a json file for further analysis.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "(1)\n\"Connecting research to potential use allows researchers to shape their work taking into account the expressed needs of key stakeholders, such as professional fact-checkers (Nakov et al., 2021).\" --> Overcomplicated sentence: \n\n(2)\n\"In this paper, we investigate narratives about intended use in automated [...]\" --> \"uses\" or \"the intended use\"\n\n(2)\n\"Based on this, we give recommendations to clarify discussions of intended use:\" --> weird style\n\n(2)\n\"[...] time, rendering resulting artefacts diffi- 122\ncult to use in real-world scenarios.\" --> style\n\n(8)\n\"We investigate narratives of intended use in fact-checking papers, finding that the majority describe how this tool will function in vague or unrealistic terms\" --> \"[...] papers, and find that [...] how a tool [...].\"\n\n(9)\n\"We chose to understand automated fact-checking artefacts through their intended uses. This is only one way understand technologies.\" --> \"[...] one way to understand technologies.\"\n\n(23)\nFigure 7 should be formatted in a cleaner way."}}, {"review_id": "yDIC18bpq3", "reviewer": "Reviewer_XbZ4", "summary": "", "questions": "A: For unsupported ends, the existence of counter evidence is given as a sufficient criterion. Is it, in general, a criterion for unsupported ends? What if there is both evidence and counter evidence? This black and white view of fact checking may be an oversimplification. Have you considered a more nuanced categorization of faithfulness to evidence, e.g. claim strength as in https://aclanthology.org/2021.emnlp-main.845/ ? ", "limitations": "", "rating": 5, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive comments, and we are very happy they found our paper important and well-positioned towards EMNLP!\n\n**Assumptions about Author Intent**\n\nWe thank the reviewer for pointing this out. This is a great point! We do not make a strong statement that automated fact-checking papers \\*always produce\\* such an artefact. However, authors of fact-checking papers often do. As we mention in lines 487-491:  *“Authors of fact-checking papers clearly believe their research to be solving a societal need: 82% of narratives in our analysis had ‘limiting misinformation' as the desired end, and an additional 7% had ‘increasing the veracity of published content’”*. The reviewer is correct that authors may have other motivations. For example, the “scientific curiosity” narrative (which appears in 8% of the papers in our analysis) is often not directly related to fighting misinformation.\n\nWe argue that if authors wish to make such claims in their papers, they should make sure there is empirical support, either by gathering evidence themselves or by citing other work that presents evidence. Unfortunately, as our paper shows, this rarely happens.\n\nWe appreciate that our findings on the prevalence of such arguments come somewhat late in the paper, appearing first in the section header for Section 7. We will bring these to the introduction in the camera-ready version if our paper is accepted.\n\n**List of papers**\n\nWe included the URL links of the papers that we analysed in the supplementary materials (the data.json file with the extracted quotes). If our paper is accepted, we will add the list to the camera-ready. The full JSON file with our annotations will also be made available on GitHub.\n\n**Analysis Statistics**\n\nWe thank the reviewer for pointing this out, we agree statistics on citations and publication year would be useful. Number of citations ranges (according to Google Scholar) from 1258 to 16. We will include a figure showing this in the camera-ready if our paper is accepted. As for the number of papers per year, we will also add the following table:\n\n| Year     | 2008 | 2009 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 |\n|----------|------|------|------|------|------|------|------|------|------|------|\n| # Papers | 1    | 1    | 1    | 4    | 5    | 11   | 23   | 15   | 28   | 11   |\n\n**Figure 1**\n\nThanks for pointing this out – we will amend the figure.\n\n**Spinoza and Heidegger**\n\nSpinoza and Heidegger both made arguments that artefacts should be understood as functions of their uses. Spinoza argued that a hatchet which does not work for its intended purpose – chopping wood – is no longer a hatchet. Heidegger went further, arguing that artefacts can only be properly understood when actively used for their intended purpose; i.e., the only way to understand a hammer is to hammer with it. We use the citations to document that some philosophers hold uses to be important for the understanding and analysis of artefacts. We will expand on this in the camera-ready if our paper is accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Important examination of the very basic goals of automated fact checking research and the extent to which they are expressed in influencial publications in the area, finding low coherence in many cases. \n\nClear, well motivated and systematic argumentation that uses just the terminology and reasoning strategies common in the fact checking literature, e.g., lacking evidence, making it well positioned toward this community.\n\nRobust and thorough annotation and analysis procedure following best practices in content analysis, resulting in both a novel categorization scheme and annotated corpus used to quantitatively back up the arguments in the paper. Exceptionally detailed account of the annotation guidelines and the meaning of each label, including examples for each single case.\n\nBroad and informative perspective on related work on the goals of fact checking from various disciplines. ", "reasons_to_reject": "Seems to be implicitly making a strong assumption, that (influencial) papers in automated fact checking always contribute an artifact positioned within some (potential) application. Contributions in NLP, on the other hand, often focus on improving or investigating specific aspects of components that can be internal to such a system, leaving out a broader discussion on the overall system, especially given space limitations. It is therefore not entirely clear whether the normative statement the authors are making, that a comprehensive narrative is always warranted, is a subjective preference or if there is evidence that it is necessary in all cases. For example, arguably it is not the researcher's job to discuss what to do with the identified misinformation after it was found by the system. \n\nNo information on the papers themselves is given in the paper, such as distribution over publication years or number of citations. It is also not specified how the number was obtained.", "questions_to_authors": "A: For unsupported ends, the existence of counter evidence is given as a sufficient criterion. Is it, in general, a criterion for unsupported ends? What if there is both evidence and counter evidence? This black and white view of fact checking may be an oversimplification. Have you considered a more nuanced categorization of faithfulness to evidence, e.g. claim strength as in https://aclanthology.org/2021.emnlp-main.845/ ? ", "paper_topic_and_main_contributions": "This review/position paper identifies common pitfalls in fact checking papers, e.g., vagueness about who should use the proposed systems and how. A comprehensive categorization of such issues is proposed and applied to the analysis of the 100 most highly cited papers in the area, using a content analysis approach conducted by the authors. Finally, recommendations are provided for improving narratives in fact checking contributions.\n\nWhile I have not worked on fact checking myself, I am familiar with the literature and the overall rationale. ", "excitement": 5, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Feasibility support does not seem to appear in figure 1. This is fine, but may be worth mentioning so the reader does not look for it.\n\nCiting Spinoza and Heidegger seems somewhat out of context given the subject of technological artifacts, which is implicitly about information technology. Expanding the discourse more explicitly may help. "}}]

Author rebuttal:

Meta-review: The decision based on the reviews suggests acceptance, but with revisions needed. The reviewers have recognized the paper's systematic research approach, clear argumentation, and its contribution towards the understanding of automated fact-checking. They find the topic relevant and timely, and appreciate the detailed annotation guidelines and analysis procedure which lend credibility to the arguments in the paper. 
However, they also identify certain limitations such as the paper's complexity and excessive use of specialized terminology which makes it difficult to understand. There was also criticism about the missing information on the papers examined. Therefore, while the paper has potential, it requires some revisions before publication.
Decision: Accept-Findings