|
--- |
|
license: afl-3.0 |
|
language: |
|
- es |
|
tags: |
|
- biomedical |
|
- social media |
|
- ner |
|
metrics: |
|
- f1 |
|
widget: |
|
- text: "La semana que viene estaremos en el I Congreso para personas con cáncer y familiares ☺ #aecc #Congreso #finde " |
|
example_title: "Oncology" |
|
- text: "No dejéis de leer esta interesantísima entrada del Dr. Martínez-Lage donde reivindica los errores médicos a la hora de diagnosticar #Alzheimer u otros tipos de #demencias." |
|
example_title: "Alzheimer" |
|
- text: "Cada vez hay más CCAA que se suman la regulación de #desfibriladores (#DESA) en espacios deportivos, lamentamos este caso de parada cardíaca que afectó de nuevo a un deportista." |
|
example_title: "cardiac arrest" |
|
- text: "La jaqueca o la migraña puede llegar a ser muy desesperante, algunas veces los remedios para dolor de cabeza de origen farmacéutico son ineficientes y por más analgésicos que tomemos el malestar no cede." |
|
example_title: "Migraine" |
|
- text: "Os sorprenderíais la de mensajes que me llegan cada día (sobre todo cuando se acerca el verano) preguntándome como eliminar la celulitis, como hacer que desaparezca mágicamente la grasita… " |
|
example_title: "Celulitis" |
|
--- |
|
|
|
# Disease mention recognizer for Spanish Social Media texts 🦠💬 |
|
This resource derives from the participation of the SINAI team in [Mining Social Media Content for Disease Mention (SocialDisNER)](https://temu.bsc.es/socialdisner/) shared task. This task focused on the recognition of disease mentions in tweets written in Spanish with the aim of using Twitter as a proxy to better understand societal perception of disease. This task brought the community effort to developing named entity recognition (NER) approaches to detect **all kinds** of disease mentions in social media text. |
|
|
|
Our approach is based on a [model pre-trained on general-domain text](https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne). In order to leverage large scale additional [Silver Standard data](https://zenodo.org/record/6803567/preview/SocialDisNER_LargeScale_additionaldata.zip#tree_item0) with automatically generated labels provided by task’s organisers we designed a two-stage fine-tuning framework. |
|
|
|
# Results |
|
The model contained in this repository constitutes the fundament of the NER system presented by the SINAI team on SocialDisNER. Enhanced with data [`pysentimiento`](https://github.com/pysentimiento/pysentimiento) pre-processing and rule-based submission post-processing, it obtained encouraging results during the official evaluation, which are summarised in the table below. |
|
|
|
| Precision | Recall | F1-score | |
|
|-----------|--------|----------| |
|
| 0.756 |0. 795 | 0.770 | |
|
|
|
|
|
# System description paper and citation |
|
[`The system description paper`](https://aclanthology.org/2022.smm4h-1.8/) was be published at Social Media Mining for Health Application (#SMM4H) held on COLING22 in October 2022. |
|
|
|
``` |
|
@inproceedings{chizhikova-etal-2022-sinai, |
|
title = "{SINAI}@{SMM}4{H}{'}22: Transformers for biomedical social media text mining in {S}panish", |
|
author = "Chizhikova, Mariia and |
|
L{\'o}pez-{\'U}beda, Pilar and |
|
D{\'\i}az-Galiano, Manuel C. and |
|
Ure{\~n}a-L{\'o}pez, L. Alfonso and |
|
Mart{\'\i}n-Valdivia, M. Teresa", |
|
booktitle = "Proceedings of The Seventh Workshop on Social Media Mining for Health Applications, Workshop {\&} Shared Task", |
|
month = oct, |
|
year = "2022", |
|
address = "Gyeongju, Republic of Korea", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://aclanthology.org/2022.smm4h-1.8", |
|
pages = "27--30", |
|
abstract = "This paper covers participation of the SINAI team in Tasks 5 and 10 of the Social Media Mining for Health ({\#}SSM4H) workshop at COLING-2022. These tasks focus on leveraging Twitter posts written in Spanish for healthcare research. The objective of Task 5 was to classify tweets reporting COVID-19 symptoms, while Task 10 required identifying disease mentions in Twitter posts. The presented systems explore large RoBERTa language models pre-trained on Twitter data in the case of tweet classification task and general-domain data for the disease recognition task. We also present a text pre-processing methodology implemented in both systems and describe an initial weakly-supervised fine-tuning phase alongside with a submission post-processing procedure designed for Task 10. The systems obtained 0.84 F1-score on the Task 5 and 0.77 F1-score on Task 10.", |
|
} |
|
``` |
