ITAINNOVA at S ocial D is NER : A Transformers cocktail for disease identification in social media in S panish (2022)

Abstract: ITAINNOVA participates in SocialDisNER with a hybrid system which combines Transformer-based Language Models (LMs) with a custom-built gazetteer for Approximate String Matching (ASM) and dedicated text processing techniques for the social media domain. Additionally, zero-shot classification capabilities have been explored in order to support different parts of the system. An extensive analysis on the interactions of these components has been accomplished, making the system stand out above the mean performance of all the participating teams.

## motivation the detection of disease mentions in tweets (so-cialdisner, @xcite is part of the smm4h shared task @xcite . socialdisner proposes a challenging task: ner (named-entity recognition) offset detection of diseases by finding the span of its mentions in tweets published in the spanish language. using social media data for health research involves facing multiple natural language processing (nlp) challenges: multilingualism, usage of formal and informal expressions, misspellings, ambiguity and so on, which may be better tackled unifying state of the art approaches and more conventional methods. in this context, itainnova participates with a hybrid system which combines transformerbased language models (lms) with a custom-built gazetteer for approximate string matching (asm) and dedicated text processing techniques for the social media domain. additionally zero-shot classification capabilities (pushp and srivastava, 2017) have been explored in order to support different parts of the system. an extensive analysis on the interactions of these components has been accomplished, making the system stand out above the mean performance of all the participating teams. ## system description the overall architecture is illustrated in figure 1 . ## results and discussion a thorough study on different state of the art transformer language models for ner in spanish, their aggregation and integration with other nlp tech-niques was conducted using the datasets of sociald-isner. the obtained results demonstrate that ensembling approaches provide the best performance, since standalone models enriched with specific rules work reasonably well on the task. the mbert with lstm-crf model outperforms other architectures, but in terms of model sizes no significant differences have been found. despite the overall strong performance, many false positives are extracted, so that further research on the effect of the pretraining corpus would be needed. using gazetteers alongside lms, when their size and quality are extensive enough, have a positive impact on the recall. however, large gazetteers are time-consuming in building and predicting phases comparing with their performance contribution. in regard to zero-shot classification, it has contributed to build gazetteers by filtering out of domain terms. nevertheless, when applied to the prediction pipeline it filters true positives and thus worsens performance, so that it has not been used on test. domain specific text processing, such as hashtag segmentation and filtering rules to remove false positives, is needed due to the linguistic complexity and orthographic diversity of social media. final results in the test set obtained with two different configurations focusing on comparativeness over gazetteer usage, are depicted in