Scientific Fact-Checking: A Survey of Resources and Approaches (2023)

Abstract: The task of fact-checking deals with assessing the veracity of factual claims based on credible evidence and background knowledge. In particular, scientific fact-checking is the variation of the task concerned with verifying claims rooted in scientific knowledge. This task has received significant attention due to the growing importance of scientific and health discussions on online platforms. Automated scientific fact-checking methods based on NLP can help combat the spread of misinformation, assist researchers in knowledge discovery, and help individuals understand new scientific breakthroughs. In this paper, we present a comprehensive survey of existing research in this emerging field and its related tasks. We provide a task description, discuss the construction process of existing datasets, and analyze proposed models and approaches. Based on our findings, we identify intriguing challenges and outline potential future directions to advance the field.

## introduction in today's digital age, vast amounts of data are generated and new scientific breakthroughs achieved at a rapid pace. with millions of scientific articles being published annually, it has become increasingly challenging for researchers and the general public to stay informed about the latest developments and discoveries across various fields. on top of that, an especially challenging task for researchers is finding appropriate evidence for scientific claims and research hypotheses they are currently investigating. exploring large academic databases and thoroughly examining scientific publications in them in order to verify specific facts is a time-consuming process. automating the process of fact-checking scientific claims using methods based on natural language processing (nlp) for knowledge exploration and evidence mining can greatly aid researchers in these efforts. one way how the internet has benefited society is by making scientific knowledge easily accessible, transferable, and searchable in a matter of seconds. inevitably, this has introduced new risks and challenges -it has become difficult to discern reliable sources from dubious content. many scientific claims found in online articles, social media posts, or news reports are not always trustworthy and backed by reliable evidence. furthermore, not only are humans prone to creating inaccurate information -modern generative language models can also produce misleading text that sounds convincing. all of these factors, combined with the quick pace at which content is proliferated online, contribute to the spread of misinformation, which has negative societal consequences @xcite . fact-checking is the task of assessing the veracity of factual claims appearing in written or spoken sources. it is traditionally performed manually by experts in journalism and dedicated applied fields. automated fact-checking appeared as an approach where methods of natural language processing (nlp) and machine learning (ml) are used to assist experts in making these decisions or completely automating the whole process @xcite . fact-checking becomes especially relevant during major political events like elections or referendums because of a sharp increase in deceptive and propagandist content. most recently, the covid-19 pandemic has brought the scientific discourse and the misinformation that comes with it into the spotlight. medical misinformation is especially dangerous because it has influenced people to try unproven cures and treatments and make harmful health-related decisions. @xcite . we define scientific fact-checking as a subset of the fact-checking task concerned with verifying the veracity of claims related to scientific knowledge. while the primary role of general fact-checking is to help detect misinformation and curb its spread, scientific fact-checking additionally aids scientists in testing their hypotheses and helps wider audiences contextualize new scientific findings. the most popular scientific domain in scientific nlp research is the biomedical domain @xcite , but insights learned from it can be generalized to other scientific domains. scientific factchecking can be performed both over the highly structured and complex language of science found in research publications and over the more easily understandable language found in news articles and online postings meant for lay audiences. many scientists have decried the misinterpretation of their work when presented in the news press @xcite , which makes scientific fact-checking even more relevant in bridging the gap between these two registers by performing an evidencebased assessment of scientific discoveries. considering the constantly increasing amount of misinformation in the digital era and the expanding number of scientific publications, the interest in developing automated fact-checking solutions and efficient resources for it is on the rise. we present this survey to systematize the existing work in this area. to the best of our knowledge, this is the first survey on fact-checking with a specific focus on the scientific domain. our three main contributions are: ## related tasks in this section, we present tasks related to scientific fact-checking. we group them into three categories: ## datasets in this section, we outline the existing datasets for scientific fact-checking that we found in the literature. the discovery process started with querying the well-known databases acl anthology, 1 ## approaches in this section, we describe different modeling approaches devised for the task of scientific factchecking. the standard framework usually consists of three major components that can all be modeled as well-established nlp tasks: document retrieval, evidence (rationale) selection, and verdict prediction @xcite . this framework is visualized in figure 1 . ## discussion in this chapter, we discuss the current challenges in scientific fact-checking and provide directions for future work and trends. evidence quality. a common challenge in factchecking is ensuring that the evidence used for making veracity decisions is appropriate and of high quality. especially in scientific fact-checking, the nature of scientific knowledge is such that it is updated and readjusted as new discoveries appear, so a claim that was once refuted by evidence could be-come supported with more substantial, more recent evidence. time-aware scoring for evidence ranking was explored for general fact-checking @xcite . additionally, scientific sources can contradict one another and give differing results for the same research hypotheses, which is related to the ml concept of learning with label disagreement @xcite . in the medical field, systematic reviews provide evidence-based clinical recommendations with the level of evidence (how much testing was performed) and the strength of recommendation (is it just a hint or a strict medical recommendation) @xcite . so far, none of the datasets have taken into account the evidence that is changing with time, disagreeing evidence, or differing levels and strength of evidence. a promising research direction is constructing resources and benchmarks that would consider these intricacies of scientific fact-checking. ## conclusion in this survey, we reviewed and systematized existing datasets and solutions for the task of scientific fact-checking. we introduced the task and compared it to its related nlp endeavors, described the existing datasets and their construction process, and explained the models used for scientific factchecking with their pipeline components. finally, we provided a critical discussion of current challenges and highlighted promising future directions for the task of scientific fact-checking.