Togedemaru at S em E val-2023 Task 8: Causal Medical Claim Identification and Extraction from Social Media Posts (2023)

Abstract: The “Causal Medical Claim Identification and Extraction from Social Media Posts task at SemEval 2023 competition focuses on identifying and validating medical claims in English, by posing two subtasks on causal claim identification and PIO (Population, Intervention, Outcome) frame extraction. In the context of SemEval, we present a method for sentence classification in four categories (claim, experience, experience_based_claim or a question) based on BioBERT model with a MLP layer. The website from which the dataset was gathered, Reddit, is a social news and content discussion site. The evaluation results show the effectiveness of the solution of this study (83.68%).

## introduction 15 one of the most brainstorming tasks facing natural 16 language processing (nlp) is the information 17 extraction @xcite ; landosi, m.y. https://causalclaims.github.io/ decision-making, can and have to be combined 31 @xcite . the aim of this paper is 32 focused on identification of causal claims 1 , 33 experience, etc., in a provided multi (or single) input (social media post as a text snippet) and the 95 requested classes to be identified for the first subtask: 96 claim(s), personal experience(s) (…) 97 question(s) claim(s) based on personal experience(s) figure 2.2 -causal claim identification -question(s), 98 claim(s) based on personal experience(s) (cont.) 99 an extracting script foot_2 shared by the task's 100 organizers was used to obtain a reddit collection of 101 around 5696 posts (for training) and 1425 posts (for 102 testing) which, after preprocessing and splitting by 103 annotations, implied around 21652 sentences (for 104 training) and 13567 sentences (for testing). the 105 initial labels have been changed into integers in the 106 interest of an easier manipulation. (0 -"claim", 1 -107 "per_exp", 2 -"question", 3 -"claim_per_exp"). 114 115 0 ## system overview the strategy picked for this task represents a causal approach to bert (biobert + mlp (figure 3 ) as found in the architecture @xcite . the starting point in the development of this solution was the model-generating code made public by yu, b., li, y. and wang, j. (2019), on github. the only significant changes were done to the labels, from 0 -"none", 1 -"causal", 2 -"cond", 3 -"corr" to 0 -"claim", 1 -"per_exp", 2 -"question", 3 -"claim_per_exp". + mlp layer; preprocessing class handling input 148 development 149 the datasets were constructed from the given 150 collection of post ids and annotations by automatic 151 extraction from the reddit platform, preprocessed 152 and further used for training the biobert + mlp 153 model. the training process took around 2.5 hours 154 and it implied a combination of 5 folds and 5 155 epochs. 156 the subtask provided an extraction script, 157 mentioned in the previous sections, to be used for 158 extracting the required training social posts/text 159 snippets. the given input files for training and 160 testing were only files describing the path from 161 which we had to extract the text by post id with 162 preset reddit credentials, process that took around 163 3 hours for the posts required by the task of interest (figure 4 ). the purpose was to extract the text and 165 match the substrings by the labels specified by 166 ranges of "<@xmath0. <@xmath1: 0 -"claim", 1 189 -"per_exp", 2 -"question", 3 -"claim_per_exp". task's submission -words share the same label as the 193 sentence. s1jpia,t5_2s23e,"[{""crowd-entityannotation"":{""entities"":[{""endo ffset"":858,""label"":""per_exp""," "startoffset"":661},{""endoffset"": 2213,""label"":""per_exp"",""starto ffset"":1861},{""endoffset"":2407," "label"":""per_exp"",""startoffset" ":2255},{""endoffset"":3254,""label "":""claim_per_exp"",""startoffset" ":2697},{""endoffset"":3620,""label "":""claim_per_exp"",""startoffset" ":3294},{""endoffset"":3751,""label "":""claim_per_exp"",""startoffset" ":3621},{""endoffset"":4480,""label "":""per_exp"",""startoffset"":3752 },{""endoffset"":4759,""label"":""q uestion"",""startoffset"":4482}]}}] " (…) 216 input dataset after extraction: 217 post_id, subreddit_id, stage1_labels, text s1jpia,t5_2s23e,"[{""crowd-entityannotation"":{""entities"":[{""endoffse t"":858,""label"":""per_exp"",""startof fset"":661},{""endoffset"":2213,""label "":""per_exp"",""startoffset"":1861},{" "endoffset"":2407,""label"":""per_exp"" ,""startoffset"":2255},{""endoffset"":3 254,""label"":""claim_per_exp"",""start offset"":2697},{""endoffset"":3620,""la bel"":""claim_per_exp"",""startoffset"" :3294},{""endoffset"":3751,""label"":"" claim_per_exp"",""startoffset"":3621},{ ""endoffset"":4480,""label"":""per_exp" ",""startoffset"":3752},{""endoffset"": 4759,""label"":""question"",""startoffs et"":4482}]}}]","de-