A ra F acts: The First Large A rabic Dataset of Naturally Occurring Claims (2021)

Abstract: We introduce AraFacts, the first large Arabic dataset of naturally occurring claims collected from 5 Arabic fact-checking websites, e.g., Fatabyyano and Misbar, and covering claims since 2016. Our dataset consists of 6,121 claims along with their factual labels and additional metadata, such as fact-checking article content, topical category, and links to posts or Web pages spreading the claim. Since the data is obtained from various fact-checking websites, we standardize the original claim labels to provide a unified label rating for all claims. Moreover, we provide revealing dataset statistics and motivate its use by suggesting possible research applications. The dataset is made publicly available for the research community.

## introduction fake news and misinformation are considered among the greatest threats to nations. the spread of fake news can cause manipulation in public opinion, which has adverse consequences to politics and journalism. moreover, the recent covid-19 pandemic revealed how medical misinformation could easily harm the health of people @xcite . notable development has been made in automated fact-checking systems over the past years. however, one of the main limitations of arabic automated fact-checking systems is the lack of arabic datasets. recently several arabic fact-checking websites (e.g., fatabyyano and misbar) have emerged to help combat the spread of rumors and fake news, especially over social media. they constitute a valuable resource of naturallyoccurring fact-checked claims in the arab world. the claims are annotated and verified by professional fact-checkers and journalists, making them a reliable source of information. while the verified information about those rumors is posted on those websites and their corresponding social media accounts, they are not gathered and unified as a dataset that can be used to develop automated systems. in this paper, we introduce arafacts, the first large arabic fact-checking dataset of naturallyoccurring and professionally-verified claims. we crawl 6,222 claims along with their factual labels, description, and additional metadata, from 5 different fact-checking websites, and make them available to the research community in a normalized form. the contributions of our work are two-fold: the rest of the paper is organized as follows. section 2 summarizes related work. section 3 describes the data collection process. section 4 provides data analysis. section 5 outlines some research applications, and section 6 concludes. ## related work several datasets of naturally occurring claims have been previously proposed, but mostly in english. for example, multifc dataset @xcite includes 38,918 claims and their metadata from 26 different fact-checking websites. similarly, liar dataset (wang, 2017) consists of 12,836 claims collected from politifact. foot_1 fakenewsnet @xcite is a mutli-dimensional data repository that contains 23,196 fact-checked articles from politifact and gossipcop. 3 it also includes additional social context related to the checked claims. other datasets that focus on topic-specific claims were published recently. fakecovid (shahi and nandini, 2020) dataset covers 5,182 multilingual fact-checked claims from 92 different fact-checking websites, all related to covid-19. moreover, pubhealth dataset kotonya and toni (2020) consists of 11,832 claims public healthrelated claims attained from 8 different websites. over the past few years, there has been an increased interest in task-specific arabic applications leading to the release of several arabic datasets. most of these are built for tasks such as checkworthiness of tweets, evidence retrieval, and claim verification @xcite . additionally, the arabic news stance dataset (ans) @xcite supports claim verification and stance prediction. the dataset is generated using existing arabic news titles from ant corpus @xcite . the limitation with ans is that it requires manual annotation to generate claims from news titles. @xcite address this limitation with the aranews dataset. aranews uses arabic articles from online news data to automatically generate arabic manipulated text. @xcite proposed a corpus consisting of 422 fact-checked arabic claims and documents associated with the claims. the corpus supports multiple tasks such as stance detection and factchecking. arafacts differs from their work in two ways; it covers a larger number of claims and all claims are professionally fact-checked. while several datasets were constructed for different tasks that are related to fact-checking, to the best of our knowledge, arafacts is the largest arabic dataset that leverages emerging arabic fact-checking websites as a source of annotated professionally-verified information. ## data collection in this section, we describe the process of selecting fact-checking websites, crawling the verification articles, and constructing the arafacts dataset. ## data analysis in this section, we perform further analysis on the collected claims. we also examined the original source domains of the claims from the embedded claim urls. figure 4 shows the distribution of the claims over publication time. we notice that the majority of the claims were published in 2020; this is due to the fact the most fact-checking websites in our dataset were launched in the past two years. figure 5 shows the most frequent 100 words in the claim text. we notice a mix of political figures, religious entities, and country names among others. finally, we analyze the types of extracted claims. figure 6 shows the frequency of the different types of claims (text claims, image claims, and video claims) for each fact-checking website. interestingly, the two most contributing fact-checking websites (misbar and fatabyyano) contain more visual claims than textual. ## example use cases this section provides example use cases and research problems that can be supported by arafacts. ## conclusion in this paper, we introduce arafacts, the first large arabic dataset of naturally-occurring claims covering about 6k claims published since 2016 and already annotated by professional fact-checkers from 5 different arabic fact-checking websites over several topical categories. the dataset supports many research tasks such as claim verification, claim retrieval, also evidence retrieval. we made our dataset publicly available to the research community, and we plan for periodic crawling to augment the newly verified claims from existing and new fact-checking websites. we believe that arafacts will serve as a valuable resource for future studies on fact-checking, such as fake image detection. our future work will focus on evaluating and utilizing arafacts for several fact-checking tasks to contribute the arabic-based work in that domain.