Neural Text Classification and Stacked Heterogeneous Embeddings for Named Entity Recognition in SMM 4 H 2021 (2021)

Abstract: This paper presents our findings from participating in the SMM4H Shared Task 2021. We addressed Named Entity Recognition (NER) and Text Classification. To address NER we explored BiLSTM-CRF with Stacked Heterogeneous embeddings and linguistic features. We investigated various machine learning algorithms (logistic regression, SVM and Neural Networks) to address text classification. Our proposed approaches can be generalized to different languages and we have shown its effectiveness for English and Spanish. Our text classification submissions have achieved competitive performance with F1-score of 0.46 and 0.90 on ADE Classification (Task 1a) and Profession Classification (Task 7a) respectively. In the case of NER, our submissions scored F1-score of 0.50 and 0.82 on ADE Span Detection (Task 1b) and Profession span detection (Task 7b) respectively.

## introduction the ubiquity of social media has led to massive user-generated content across various platforms. twitter is a popular micro-blogging platform that allows its users to publish tweets up to 280 characters. the common public uses twitter to share life-related personal and professional experiences with others. personal experiences often involve health-related incidents including mentions of adverse drug effect (ade); this information is crucial to study pharmacovigilance. in the context of the covid-19 pandemic, the professional experiences may include information about professions and occupations which are vulnerable due to either direct exposure to the virus or due to the associated mental health issues; detecting vulnerable occupations is critical to adopt necessary preventive measures. recent research focuses on mining twitter data for adverse drug effect detection @xcite @xcite . the distinctive style of communication on twitter presents unique challenges including informal (brief) text, misspellings, noisy text, abbreviations, data sparsity, colloquial expressions and multilinguality. ## task description and contribution we participate in the following two tasks organized by smm4h workshop 2021 @xcite : following are our multi-fold contributions: 1. to address ner tasks, we have employed a neural network based sequence classifier, i.e. bilstm-crf and investigated various heterogeneous embeddings. we further investigated the combination of character embeddings, static word embeddings and contextualized embeddings in a stacked format. we also incorporated linguistic features such as part-of-speech tags (pos), orthographic features etc. we apply the proposed modelling approaches to both english and spanish texts. in profession span detection (task 7b) our submission (team:mic-nlp) achieved the f1-score of 0.824 which is 6 points higher than the arithmetic median of all the submissions; in case of ade span detection our submission scored f1-score of 0.50, around 8 points higher than the arithmetic median of the participating submissions. ## methodology in the following sections we discuss our proposed model for named entity recognition and text classification. ## conclusion in this paper, we described our system with which we participate in task 1(adverse drug effect classification and extraction) and task 7 (identification of professions and occupations in spanish tweets) in the smm4h shared task 2021. our ner system employed stacked heterogeneous em-