Lasige- B io TM at P rof NER : B i LSTM - CRF and contextual S panish embeddings for Named Entity Recognition and Tweet Binary Classification (2021)

Abstract: The paper describes the participation of the Lasige-BioTM team at sub-tracks A and B of ProfNER, which was based on: i) a BiLSTM-CRF model that leverages contextual and classical word embeddings to recognize and classify the mentions, and ii) on a rule-based module to classify tweets. In the Evaluation phase, our model achieved a F1-score of 0.917 (0,031 more than the median) in sub-track A and a F1-score of 0.727 (0,034 less than the median) in sub-track B.

## introduction the track "profner-st: identification of professions & occupations in health-related social media" @xcite ) occurred in the context of the "social media mining for health applications (#smm4h) shared task 2021" @xcite , and included two different sub-tracks that focused on spanish twitter data: this paper describes the participation of the lasige-biotm team in the aforementioned subtracks. we applied 8 different models ner models (4 supervised models based on bilstm-crf architecture, 3 rule-based models) to predict entities for sub-track b and explored the impact of performing data augmentation in the training set. for sub-track a, we developed a rule-based model for tweet classification that was based on the ner output for sub-track b. ## conclusion during the practice phase, we explored different approaches to participate in sub-tracks a e b of profner: data augmentation on training set, and application of mer and a bilstm-crf model for ner and further tweet classification. for the evaluation phase we applied the bilstm-crf model on the test set of profner corpus and achieved f1scores of 0.917 (0,031 more than the median) and in sub-track our model achieved a f1-score of 0.727 (0,034 less than the median). the code to run the experiments is available in our github page 1 . for future work, we intend to perform hyper-parameter optimisation for the bilstm-crf model, such as learning rate, hidden size, and specially the number of training epochs, since we had limited available time to perform the training of the model. we will also explore the use of different contextualised embeddings, since the models using this type of embeddings seem to achieve better performance compared to those using classical word embeddings. besides, to improve tweet classification we will explore the application of named entity linking tools @xcite to link the recognized entities in sub-track b to structured vocabularies that contain hierarchical relationships between concepts, such as mesh or dbpedia. this way, it will be possible to know the ancestors for a given entity, which will provide the context to effectively determine if the entity is associated with an occupation or not.