PANACEA : An Automated Misinformation Detection System on COVID -19 (2023)

Abstract: In this demo, we introduce a web-based misinformation detection system PANACEA on COVID-19 related claims, which has two modules, fact-checking and rumour detection. Our fact-checking module, which is supported by novel natural language inference methods with a self-attention network, outperforms state-of-the-art approaches. It is also able to give automated veracity assessment and ranked supporting evidence with the stance towards the claim to be checked. In addition, PANACEA adapts the bi-directional graph convolutional networks model, which is able to detect rumours based on comment networks of related tweets, instead of relying on the knowledge base. This rumour detection module assists by warning the users in the early stages when a knowledge base may not be available.

## introduction the dangers of misinformation have become even more apparent to the general public during the covid-19 pandemic. following false treatment information has led to a high number of deaths and hospitalisations @xcite . manual verification can not scale to the amount of misinformation being spread, therefore there is a need to develop automated tools to assist in this process. in this work, we focus on automating misinformation detection using information from credible sources as well as social media. we produce a webbased tool that can be used by the general public to inspect relevant information about the claims that they want to check, see supporting or refuting evidence, and social media propagation patterns. for false information, the commonly used and relatively reliable method for automated veracity assessment is to check the claim against a verified knowledge base, which we call fact-checking. previous works such as evidenceminer @xcite , pubmed 1 and covid-19 fact-1 https://www.ncbi.nlm.nih.gov/pmc/ checking sites recommended by the nhs foot_0 are all designed to retrieve related documents/sentences from a reliable knowledge base. however, this approach leaves users to summarise a large amount of potentially conflicting evidence themselves. panacea, which is supported by novel natural language inference methods @xcite , is instead able to provide automated veracity assessment and supporting evidence for the input claim. in addition, previous works retrieve results using entities in the input claim, and thus often include results related to a keyword in the input claim instead of the whole query, while panacea considers the whole query for better result. the supporting pieces of evidence are also ranked by their relevance score and classified according to their stance towards the input claim. in addition to false information, truthful information can also be misused to harm competitors or gain attention on social media @xcite . however, the latter is harder to be found by checking reliable knowledge bases as those are focused on false information. regarding this issue, previous work has analysed the spread of misinformation using features such as stance @xcite , sentiment, topics, geographical spread, the reliability of external links included in the tweet @xcite , origin and propagation networks @xcite . however, it is still hard for users to identify rumours by directly looking at those features. previous research shows that the propagation pattern is different between fake and real news, which would offer additional features for early detection of misinformation on social media @xcite . panacea extends this by using tweets' propagation patterns to identify rumours. rumour detection is not as reliable as fact-checking, but it generalises the system to various situations that fact-checking cannot cover: first, true or unverified information with intent to harm; second, scenarios where no verified knowledge database is available. rumour detection cannot prove the truth of a claim but may alert the user about claims with a high risk of being misinformation. previous work have either retrieved tweets from a short fixed time period @xcite or search recent tweets @xcite , which is limited by twitter to only the last 7 days. we instead maintain an updated database which is constituted of an annotated tweets dataset with popular claims and an unlabelled streaming of covid-19 related tweets that are crawled and selected periodically to update the dataset. besides building on the various analytic functionalities used in previous work, panacea improves the architecture of these elements and adds extra features to the updated dataset for more efficient results. a screencast video introducing the system foot_1 , illustrating its use in the checking of a covid-19 claim, and the demo foot_2 are also available online. the system can be easily adapted to other claim topics. panacea covers various types of misinformation detection related to covid-19 with the following contributions: ## datasets the following datasets are used in the project: knowledge database this is used for factchecking, and includes covid-19 related documents from selected reliable sources foot_3 . the documents were cleaned and split into 300 token paragraphs to construct a reliable knowledge database, whose supporting documents are retrieved and visualised in our system. panacea dataset (arana-catania et al., 2022), constructed from covid-19 related data sources foot_4 and using bm25 and monot5 @xcite to remove duplicate claims. this dataset includes 5,143 labelled claims (1,810 false and 3,333 true), and their respective text, source and claim sub-type. ## evaluation results fact-checking we investigate the performance of our system in document retrieval and veracity assessment in @xcite . ## conclusion this paper introduces a web-based system on factchecking and rumour detection based on novel natural language processing models for covid-19 misinformation detection. going forward, we will keep updating the data and explore other methods for misinformation identification to improve the current system and introduce more functions to the system as part of our continuing efforts to support the general public to identify misinformation.