Detecting False Claims in Low-Resource Regions: A Case Study of C aribbean Islands (2022)

Abstract: The COVID-19 pandemic has created threats to global health control. Misinformation circulated on social media and news outlets has undermined public trust towards Government and health agencies. This problem is further exacerbated in developing countries or low-resource regions, where the news is not equipped with abundant English fact-checking information. In this paper, we make the first attempt to detect COVID-19 misinformation (in English, Spanish, and Haitian French) populated in the Caribbean regions, using the fact-checked claims in the US (in English). We started by collecting a dataset of Caribbean real & fake claims. Then we trained several classification and language models on COVID-19 in the high-resource language regions and transferred the knowledge to the Caribbean claim dataset. The experimental results of this paper reveal the limitations of current fake claim detection in low-resource regions and encourage further research on multi-lingual detection.

## introduction in this work, we refer to false claim as assertions that are not supported by facts and are made with the objective of misleading or deceiving the public @xcite . social media platforms enable people to independently publish and share media content without scrutiny filters for credibility and integrity foot_0 . therefore, inaccurate, false, malicious, and propagandistic content have become abundant in social media. furthermore, when false claims travel across regions and often get translated/modified, it becomes increasingly difficult for machine learning (ml) models to detect such false claims. online surveillance (i.e., false claim detectors) systems are often primarily pre-trained on high-resource languages (e.g., english, chinese). despite significant progress in ml models, however, building and maintaining ml models in lowresource languages (e.g., tagalog, haitian creole) are still challenging due to its scarce data or language lexicon and translation barriers which are indigenous to low-resource language settings. this poses a natural question: "how effective are computational ml solutions developed in high-resource regions to detect false claims circulating in low-resource regions?" in this paper, to answer this question, we propose the first thorough case study on the detection of false claims in the caribbean islands. fact-checking initiatives are scarce and inept in low-resource settings, especially for the caribbean islands due to the cultural and linguistically diverse nature of their languages. the caribbean region is a developing, heterogeneous, interconnected archipelago that is vulnerable to false claims campaigns. it consists of 35 states and territories bordering the gulf of mexico and caribbean sea foot_1 . the caribbean has six official languages: spanish, english, french, and dutch, as well as two indigenous creoles (haitian creole and papiamento) 3 . our data curation initiative shows that this region lacks essential technological resources and infrastructure to combat false claim propagation. few fact-checking organizations exist, and they have limited data covering the caribbean. major news outlets such as loop news make significant efforts to debunk false claims. these initiatives are essential but inadequate to effectively respond to prevailing false claims during crises. in particular, we studied two research questions: rq1: how do ml models trained in high-resource languages perform with current caribbean false claims? rq2: are more sophisticated ml techniques (e.g., transfer learning), useful to detect false claims in the caribbean? note that the focus of our investigation is on the covid-19 related false claims in the caribbean islands. ml models trained in high-resource languages are not easily transferable to low-resource languages. one of the main challenges comes from data scarcity (i.e., lack of labeled training data in low-resource languages). this issue is further exacerbated by the application of false claims detection that suffers from imbalance (i.e., where the number of labeled false claims is significantly smaller than that of labeled true claims). therefore, to thoroughly study false claims in the caribbean islands, more sophisticated ml techniques that address indigenous nuances need to be tested. ## related work since the onset of the covid-19 pandemic, misinformation in different languages has been circulating on social media. the covid-19 misinformation datasets can be roughly divided into two categories: monolingual and multilingual. coaid (cui and lee, 2020), recovery @xcite , cmu-miscov19 (memon and carley, 2020), checked @xcite and costraint task dataset @xcite are monolingual datasets in high-resource languages (english or chinese). coaid is a diverse covid-19 misinformation dataset, including 5,216 news about covid-19, and ground truth labels. multilingual datasets contain news pieces in multiple languages. mm-covid @xcite ) contains false & real news content in 6 different languages. fakecovid (shahi and nandini, 2020) has 5,182 covid-19 fact-checking news pieces in 40 languages. with the urge to combat the infodemic in developing countries or immigrant communities speaking low-resource languages, researchers have been studying how to transfer the pre-trained models on high-resource domains to low resource domains. @xcite proposed a cross-lingual false claims detector called "crossfake", which is trained based on a high-resource language (english) covid-19 news corpus and used to pre-dict news credibility in a low-resource language (chinese). @xcite proposed two model generalization methods on covid-19 fake news for more robust fake news detection in different covid-19 misinformation datasets. in this paper, we chose the false claim detection in the caribbean region as a showcase. it is a challenging problem due to the multiculturalism and multilingualism of caribbean people. we studied how to leverage the pre-trained models from high-resource regions (coaid) to detect misinformation in a lowresource region (caribbean false claim data). ## conclusion high-resource detection models have low accuracy with classifying caribbean false claims data. region-specific data challenges have shown to reduce the performance of high-resource ml models. this encourages the use of sophisticated ml techniques and ai methodologies to capture signals that current models are unable to recognize. our experiment with transfer learning has shown improvements with ml models' performance. the findings in this research support our hypothesis: high-resource language model performs poorly on low-resource language data. future studies need to focus efforts on improving false claims detection in the caribbean. a major challenge is that every island has its unique creole, which complicates ml models trained in formal settings. since the jamaican languages are a combination of several languages, even the best language translator are ineffective in accurately translating the language to english. this poses another difficulty to the problem of false claims detection. false claims are the greatest threat to public health in the caribbean and globally. as we saw with covid19, if we do not address false claims, epidemic/pandemic diseases will spread exponentially @xcite .