Identifying professions & occupations in Health-related Social Media using Natural Language Processing (2021)

Abstract: This paper describes the entry of the research group SINAI at SMM4H’s ProfNER task on the identification of professions and occupations in social media related with health. Specifically we have participated in Task 7a: Tweet Binary Classification to determine whether a tweet contains mentions of occupations or not, as well as in Task 7b: NER Offset Detection and Classification aimed at predicting occupations mentions and classify them discriminating by professions and working statuses.

## related work social media plays an important role where people can share information related to health. this information can be used for public health monitoring tasks through the use of nlp techniques. on one hand, in terms of document classification in the medical field, many researchers have used social networks as a source of information to develop and evaluate systems. for example, to predict mental illnesses such as depression or anorexia @xcite and to detect nonmedical prescription medication @xcite . more recently, new studies analyzed health, psychosocial, and social issues emanating from the covid-19 pandemic from social network comments using nlp @xcite @xcite . on the other hand, several ner systems have been developed using nlp-based systems such as medlee @xcite 2020 ) and the chemdner challenge @xcite . finally, smm4h ( weissenbacher et al., 2019) ## dataset organizers provided us with a dataset consisting of 8,000 tweets from twitter subdivided into two subsets: 6,000 tweets with which to train our systems and 2,000 tweets to validate them. namely, train set and dev set, accordingly. besides the tweet's identifier and text span, a binary value was used to determine whether it contained a profession or not as well as its corresponding annotated entities tagged using the insideoutside-beginning (ibo) format. to compare the performance of all the systems presented at this shared task, we were provided with another dataset, namely test set, consisting of 2,000 processed tweets and 25,000 raw tweets for the background set. ## methodology 129 for our participation we employ ml models en202 5.1 task 7a: binary classification 203 we submitted 2 runs for the evaluation phase: a 204 combination of svm (svm+dic) and an bilstm 205 model (bilstm+dic), both combined with our 206 professions dictionary, the scoring associated to 207 these systems applied to the provided datasets is 208 displayed in while the performance of our systems on the training dataset was, at average, close to 0.74 (f1), on the test set it was decreased in a 21%. therefore, resulting in a value close to 0,59 (again, f1), 31% below the median. ## conclusion profner shared task on identifying professions and occupations we implemented three systems. first two are aimed at the binary classification task (task a) using an svm and a bilstm approach, both combined with our professions dictionary. the latter system follows a crf approach combined with our professions and working statuses dictionaries and it is applied to the ner task (task b). our predictions for the training set were consistent with those obtained on the test set for the second task (ner) whereas our approaches for the first task (binary classification) fell short of our expectations by 21% below our training results. for future work, we will use the gold test in order to perform a deeper analysis to assess why did this event happened and therefore improve the performance of our systems.