The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Who (2023)

Abstract: Automated fact-checking is often presented as an epistemic tool that fact-checkers, social media consumers, and other stakeholders can use to fight misinformation. Nevertheless, few papers thoroughly discuss how . We document this by analysing 100 highly-cited papers, and annotating epistemic elements related to intended use, i.e., means, ends, and stakeholders. We find that narratives leaving out some of these aspects are common, that many papers propose inconsistent means and ends, and that the feasibility of suggested strategies rarely has empirical backing. We argue that this vagueness actively hinders the technology from reaching its goals, as it encourages overclaiming, limits criticism, and prevents stakeholder feedback. Accordingly, we provide several recommendations for thinking and writing about the use of fact-checking artefacts.

## introduction following an increased public interest in online misinformation and ways to fight it, fact-checking has become indispensable @xcite . this has been matched by a corresponding surge in nlp work tackling automated fact-checking and related tasks, such as rumour detection and deception detection @xcite . in the process, many models, datasets, and applications have been created, referred to as artefacts. a key part of technological artefacts are their intended uses @xcite . some have argued that artefacts can only be fully understood through these @xcite . @xcite argued that a hatchet which does not work for its intended purpose -chopping wood -is no longer a hatchet. @xcite went further, arguing that artefacts can only be properly understood when actively used for their intended purpose; i.e., the only way to understand a hammer is to hammer with it. in this past election cycle for the 45th president of the united states, the world has witnessed a growing epidemic of fake news. the plague of fake news not only poses serious threats to the integrity of journalism, but has also created turmoils in the political world. the worst realworld impact @xcite . . . @xcite are the first to release a public fake news detection and fact-checking dataset, but it only includes 221 statements, which does not permit machine learning based assessments. to address these issues, we introduce the liar dataset, which includes 12,836 short statements labeled for truthfulness, subject, context/venue, speaker, state, party, and prior history. figure 1 : annotated quotes from @xcite . we highlight the motivation of the work in blue , and the model means (classification) in yellow . the goal stated (epistemic ends) is to limit misinformation. however, the authors do not state the data actors (who) and the application means (how) for reaching this goal. automated fact-checking artefacts are no different. the majority of papers envision them as epistemic tools to limit misinformation. in our analysis, we find that 82 out of 100 automated fact-checking papers are motivated as such. unfortunately, many papers only discuss how this will be achieved in vague terms -authors argue that automated factchecking will be used against misinformation, but not how or by whom (see figure 1 ). connecting research to potential use allows researchers to shape their work in ways that take into account the expressed needs of key stakeholders, such as professional fact-checkers @xcite . it also enables critical work @xcite , and facilitates thinking about unintended shortcomings, e.g., dual use @xcite to triage claims (application means) made by politicians (data subjects), in order to limit misinformation (ends)." this constitutes a narrative, the efficacy of which may have feasibility support (e.g., a user study). @xcite . @xcite argue that "people who knowingly design, develop, deploy, or use a computing artifact can do so responsibly only when they make a reasonable effort to take into account the sociotechnical systems in which the artifact is embedded". underspecification risks overclaiming, as authors cannot estimate if the aim they claim to work towards can actually be met by the means they employ. for automated fact-checking this is especially problematic. overclaiming can create misinformation about misinformation, resulting in the opposite effect from the one intended @xcite . we put forward the notion that the development of automated fact-checking into a useful tool for fighting misinformation will be more constructive with a clearer vision of intended uses. in this paper, we investigate narratives about intended use in automated fact-checking papers. we focus on those where fact-checking artefacts are introduced as epistemic tools, i.e., where the aim is to increase the quality or quantity of knowledge possessed by people. we analyse 100 highly cited papers on the topic. we manually annotate the epistemic elements of their narratives (shown in figure 2 ), relying on content analysis @xcite . specifically, we extract the stakeholders whose epistemic experience will be affected (i.e., data subjects and data actors), the strategies the authors propose (i.e., application and model means), and their intended goals (i.e., ends). we categorise the narratives extracted by analysing the links between these elements. we find that many papers contain narratives with similar types of vagueness, leaving out key details such as the actors involved. based on this, we give recommendations to clarify discussions of intended use: 1) clearly state data actors, 2) clearly state application means, 3) account for stakeholders beyond journalists, 4) ensure that the epistemic narratives are coherent and supported by relevant literature, and 5) empirically study the effectiveness of the proposed uses. ## related work a series of recent studies investigate the values and practices of nlp and machine learning research. @xcite argues that current practices in machine learning often follow historically and socially held norms, conventions, and stereotypes when they make predictions. @xcite further study the values encoded in ml research by annotating influential papers, finding that, while research is often framed as neutral, resulting artefacts can deeply affect institutions and communities. several papers have recently examined assistive uses of nlp. @xcite found that users frequently "overrely" on ai/ml tools, accepting suggestions even when they are wrong without double-checking. consequently, @xcite found that coding assistants can lead to insecure code. however, @xcite showed that explanations could reduce this effectalbeit only so long as the explanations are designed to be easily understandable and verifiable by the user. similar to our criticism of research in automated fact-checking, gooding (2022) argued that, for text simplification, the disconnect between algorithms and their applications hindered discussions of ethics. in automated fact-checking, nakov et al. (2021); @xcite have recently attempted to connect research to users by examining the concerns of professional fact-checkers. generally, they find that organisations desire automation beyond veracity prediction models, which cannot function in isolation of media monitoring, claim detection, and evidence retrieval. along parallel lines, @xcite found that many automated fact-checking datasets make unrealistic assumptions about what information is available at inference time, limiting the usefulness of the resulting artefacts (i.e., systems and datasets) in real-world scenarios. search engines have previously been studied as epistemic tools, with findings generalising to automated fact-checking. @xcite argued that search engines perform the roles of surrogate experts, connecting information seekers to testimony, and choosing how to prioritise testimonial authorities. @xcite further argued that search engine providers, developers, and researchers therefore hold responsibility for beliefs formed by users, as the internal workings of systems are not accessible to users. there are documented real-world consequences from this opacity, e.g., @xcite found evidence of bias in journalistic publications due to overreliance on biased ranking algorithms. ## content analysis to understand the narratives of the fact-checking literature, we perform a content analysis following bengtsson (2016); @xcite by means of annotating 100 highly-cited papers. as recommended by @xcite , we annotated the data in two rounds, working inductively in the first round. that is, during the first pass we began with a small set of labels for each element, and added new as necessary. we then unified the two sets of labels (i.e., the old and new sets of labels), merging labels where necessary. for the second pass, we reannotated each paper with labels from the unified list, adding no new labels. ## extracted epistemic elements we extracted the following lists of labels for epistemic elements, along with the percentage of narratives they appeared in. note that the annotation is multilabel, so the percentages do not sum to 1. definitions of each label can be found in appendix b.2. data subjects social media users (24.9%), professional journalists (15%), politicians & public figures (5.6%), product reviewers (1.7%), technical writers (1.7%), citizen journalists (0.4%). ## extracted narratives at the discourse level, we extracted epistemic narratives. below, we give a short definition of each. we also calculated the percentage of papers each narrative appears in -see figure 4 . note that this annotation is also multilabel, so the percentages do not sum to 1. figure 3 : examples of vague narratives in highly cited automated fact-checking papers. ## findings based on our content analysis, we identify several areas of concern in the analysed papers. in the following sections, we discuss our main findings. ## recommendations authors of fact-checking papers clearly believe their research to be solving a societal need: 82% of narratives in our analysis had "limiting misinformation" as the desired end, and an additional 7% had "increasing the veracity of published content". as we have argued, vagueness around intended use in current papers may prevent the community from contributing to these goals, as it can block input from key stakeholders or obscure discussion of risks. our recommendations below focus on bringing research on automated fact-checking into closer alignment with its stated goal. we emphasise that our intention is not to create another checklist for nlp conferences, as clarity surrounding intended use is already covered in the recently adopted arr responsible nlp checklist foot_1 @xcite . however, authors may find our recommendations useful for meeting that requirement. ## conclusion researchers working on automated fact-checking envision an epistemic tool for the fight against misinformation. we investigate narratives of intended use in fact-checking papers, finding that the majority describe how this tool will function in vague or unrealistic terms. this limits discussion of possible harms, prevents researchers from taking feedback from relevant stakeholders into account, and ultimately works against the possibility of using this technology to meet the stated goals. along with our analysis, we give five recommendations to help researchers avoid vagueness. we emphasise the need for clearly discussing data actors and application means, including groups beyond professional journalists as stakeholders, and ensuring that narratives are coherent and backed by relevant literature or empirical results. although we have focused on automated fact-checking in this paper, we hope our recommendations can inspire similar introspection in related fields, such as hate speech detection.