Claim Detection in Biomedical T witter Posts (2021)

Abstract: Social media contains unfiltered and unique information, which is potentially of great value, but, in the case of misinformation, can also do great harm. With regards to biomedical topics, false information can be particularly dangerous. Methods of automatic fact-checking and fake news detection address this problem, but have not been applied to the biomedical domain in social media yet. We aim to fill this research gap and annotate a corpus of 1200 tweets for implicit and explicit biomedical claims (the latter also with span annotations for the claim phrase). With this corpus, which we sample to be related to COVID-19, measles, cystic fibrosis, and depression, we develop baseline models which detect tweets that contain a claim automatically. Our analyses reveal that biomedical tweets are densely populated with claims (45 % in a corpus sampled to contain 1200 tweets focused on the domains mentioned above). Baseline classification experiments with embedding-based classifiers and BERT-based transfer learning demonstrate that the detection is challenging, however, shows acceptable performance for the identification of explicit expressions of claims. Implicit claim tweets are more challenging to detect.

## introduction social media platforms like twitter contain vast amounts of valuable and novel information, and biomedical aspects are no exception @xcite . doctors share insights from their everyday life, patients report on their experiences with particular medical conditions and drugs, or they discuss and hypothesize about the potential value of a treatment for a particular disease. this information can be of great value -governmental administrations or pharmaceutical companies can for instance learn about unknown side effects or potentially beneficial off-label use of medications. at the same time, unproven claims or even intentionally spread misinformation might also do great harm. therefore, contextualizing a social media message and investigating if a statement is debated or can actually be proven with a reference to a reliable resource is important. the task of detecting such claims is essential in argument mining and a prerequisite in further analysis for tasks like factchecking or hypotheses generation. we show an example of a tweet with a claim in figure 1 . claims are widely considered the conclusive and therefore central part of an argument @xcite , consequently making it the most valuable information to extract. argument mining and claim detection has been explored for texts like legal documents, wikipedia articles, essays @xcite @xcite , social media and web content @xcite @xcite . it has also been applied to scientific biomedical publications @xcite , but biomedical arguments as they occur on social media, and particularly twitter, have not been analyzed yet. with this paper, we fill this gap and explore claim detection for tweets discussing biomedical topics, particularly tweets about covid-19, the measles, cystic fibrosis, and depression, to allow for drawing conclusions across different fields. our contributions to a better understanding of biomedical claims made on twitter are, (1), to publish the first biomedical twitter corpus manually labeled with claims (distinguished in explicit and implicit, and with span annotations for explicit claim phrases), and (2), baseline experiments to detect (implicit and explicit) claim tweets in a classification setting. further, (3), we find in a cross-corpus study that a generalization across domains is challenging and that biomedical tweets pose a particularly difficult environment for claim detection. ## related work detecting biomedical claims on twitter is a task rooted in both the argument mining field as well as the area of biomedical text mining. ## corpus creation and analysis as the basis for our study, we collect a novel twitter corpus in which we annotate which tweets contain biomedical claims, and (for all explicit claims) which tokens correspond to that claim. ## methods in the following sections we describe the conceptual design of our experiments and introduce the models that we use to accomplish the claim detection task. ## discussion and future work in this paper, we presented the first data set for biomedical claim detection in social media. in our first experiment, we showed that we can achieve an acceptable performance to detect claims when the distinction between explicit or implicit claims is not considered. in the cross-domain experiment, we see that text formality, which is one of the main distinguishing feature between the two corpora, might be an important factor that influences the level of difficulty in accomplishing the claim detection task. our hypothesis in this work was that biomedical information on twitter exhibits a challenging setting for claim detection. both our experiments indicate that this is true. future work needs to investigate what might be reasons for that. we hypothesize that our twitter dataset contains particular aspects that are specific to the medical domain, but it might also be that other latent variables lead to confounders (e.g., the time span that has been used for crawling). it is important to better understand these properties. we suggest future work on claim detection models optimize those to work well across domains. to enable such research, this paper contributed a novel resource. this resource could further be improved. one way of addressing the moderate agreement between the annotators could be to include annotators with medical expertise to see if this ultimately facilitates claim annotation. additionally, a detailed introspection of the topics covered in the tweets for each disease would be interesting for future work since this might shed some light on which topical categories of claims are particularly difficult to label. the covid-19 pandemic has sparked recent research with regards to detecting misinformation and fact-checking claims (e.g., @xcite or @xcite ). exploring how a claimdetection-based fact-checking approach rooted in argument mining compares to other approaches is up to future research.