pwc-archive/papers-with-abstracts huggingface dataset:
KFU NLP Team at SMM4H 2021 Tasks: Cross-lingual and Cross-modal BERT-based Models for Adverse Drug Effects
Authors: Andrey Sakhovskiy, Zulfat Miftahutdinov, Elena Tutubalina
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Classification, Task 2

This paper describes neural models developed for the Social Media Mining for Health (SMM4H) 2021 Shared Task. We participated in two tasks on classification of tweets that mention an adverse drug effect (ADE) (Tasks 1a & 2) and two tasks on extraction of ADE concepts (Tasks 1b & 1c). For classification, we investigate the impact of joint use of BERTbased language models and drug embeddings obtained by chemical structure BERT-based encoder. The BERT-based multimodal models ranked first and second on classification of Russian (Task 2) and English tweets (Task 1a) with the F1 scores of 57% and 61%, respectively. For Task 1b and 1c, we utilized the previous year’s best solution based on the EnDR-BERT model with additional corpora. Our model achieved the best results in Task 1c, obtaining an F1 of 29%.
Transformer-based Multi-Task Learning for Adverse Effect Mention Analysis in Tweets
Authors: George-Andrei Dima, Dumitru-Clementin Cercel, Mihai Dascalu
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Multi-Task Learning

This paper presents our contribution to the Social Media Mining for Health Applications Shared Task 2021. We addressed all the three subtasks of Task 1: Subtask A (classification of tweets containing adverse effects), Subtask B (extraction of text spans containing adverse effects) and Subtask C (adverse effects resolution). We explored various pre-trained transformer-based language models and we focused on a multi-task training architecture. For the first subtask, we also applied adversarial augmentation techniques and we formed model ensembles in order to improve the robustness of the prediction. Our system ranked first at Subtask B with 0.51 F1 score, 0.514 precision and 0.514 recall. For Subtask A we obtained 0.44 F1 score, 0.49 precision and 0.39 recall and for Subtask C we obtained 0.16 F1 score with 0.16 precision and 0.17 recall.
Pre-trained Transformer-based Classification and Span Detection Models for Social Media Health Applications
Authors: Yuting Guo, Yao Ge, Mohammed Ali Al-Garadi, Abeed Sarker
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Classification

This paper describes our approach for six classification tasks (Tasks 1a, 3a, 3b, 4 and 5) and one span detection task (Task 1b) from the Social Media Mining for Health (SMM4H) 2021 shared tasks. We developed two separate systems for classification and span detection, both based on pre-trained Transformer-based models. In addition, we applied oversampling and classifier ensembling in the classification tasks. The results of our submissions are over the median scores in all tasks except for Task 1a. Furthermore, our model achieved first place in Task 4 and obtained a 7% higher F1-score than the median in Task 1b.
BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter
Authors: Alham Fikri Aji, Made Nindyatama Nityasya, Haryo Akbarianto Wibowo, Radityo Eko Prasojo, Tirana Fatyanosa
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Hyperparameter Optimization

This paper describes our team’s submission for the Social Media Mining for Health (SMM4H) 2021 shared task. We participated in three subtasks: Classifying adverse drug effect, COVID-19 self-report, and COVID-19 symptoms. Our system is based on BERT model pre-trained on the domain-specific text. In addition, we perform data cleaning and augmentation, as well as hyperparameter optimization and model ensemble to further boost the BERT performance. We achieved the first rank in both classifying adverse drug effects and COVID-19 self-report tasks.
UACH-INAOE at SMM4H: a BERT based approach for classification of COVID-19 Twitter posts
Authors: Alberto Valdes, Jesus Lopez, Manuel Montes
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Binary Classification

This work describes the participation of the Universidad Autónoma de Chihuahua - Instituto Nacional de Astrofísica, Óptica y Electrónica team at the Social Media Mining for Health Applications (SMM4H) 2021 shared task. Our team participated in task 5 and 6, both focused on the automatic classification of Twitter posts related to COVID-19. Task 5 was oriented on solving a binary classification problem, trying to identify self-reporting tweets of potential cases of COVID-19. Task 6 objective was to classify tweets containing COVID-19 symptoms. For both tasks we used models based on bidirectional encoder representations from transformers (BERT). Our objective was to determine if a model pretrained on a corpus in the domain of interest can outperform one trained on a much larger general domain corpus. Our F1 results were encouraging, 0.77 and 0.95 for task 5 and 6 respectively, having achieved the highest score among all the participants in the latter.
System description for ProfNER - SMMH: Optimized finetuning of a pretrained transformer and word vectors
Authors: David Carreto Fidalgo, Daniel Vila-Suero, Francisco Aranda Montes, Ignacio Talavera Cepeda
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Language Modeling, Language Modelling

This shared task system description depicts two neural network architectures submitted to the ProfNER track, among them the winning system that scored highest in the two sub-tasks 7a and 7b. We present in detail the approach, preprocessing steps and the architectures used to achieve the submitted results, and also provide a GitHub repository to reproduce the scores. The winning system is based on a transformer-based pretrained language model and solves the two sub-tasks simultaneously.
Word Embeddings, Cosine Similarity and Deep Learning for Identification of Professions & Occupations in Health-related Social Media
Authors: Sergio Santamaría Carrasco, Roberto Cuervo Rosillo
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Decoder, Word Embeddings

ProfNER-ST focuses on the recognition of professions and occupations from Twitter using Spanish data. Our participation is based on a combination of word-level embeddings, including pre-trained Spanish BERT, as well as cosine similarity computed over a subset of entities that serve as input for an encoder-decoder architecture with attention mechanism. Finally, our best score achieved an F1-measure of 0.823 in the official test set.
Classification, Extraction, and Normalization : CASIA_Unisound Team at the Social Media Mining for Health 2021 Shared Tasks
Authors: Tong Zhou, Zhucong Li, Zhen Gan, Baoli Zhang, Yubo Chen, Kun Niu, Jing Wan, Kang Liu, Jun Zhao, Yafei Shi, Weifeng Chong, Shengping Liu
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Language Modeling, Language Modelling

This is the system description of the CASIA_Unisound team for Task 1, Task 7b, and Task 8 of the sixth Social Media Mining for Health Applications (SMM4H) shared task in 2021. Targeting on deal with two shared challenges, the colloquial text and the imbalance annotation, among those tasks, we apply a customized pre-trained language model and propose various training strategies. Experimental results show the effectiveness of our system. Moreover, we got an F1-score of 0.87 in task 8, which is the highest among all participates.
BERT based Adverse Drug Effect Tweet Classification
Authors: Tanay Kayastha, Pranjal Gupta, Pushpak Bhattacharyya
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Classification

This paper describes models developed for the Social Media Mining for Health (SMM4H) 2021 shared tasks. Our team participated in the first subtask that classifies tweets with Adverse Drug Effect (ADE) mentions. Our best performing model utilizes BERTweet followed by a single layer of BiLSTM. The system achieves an F-score of 0.45 on the test set without the use of any auxiliary resources such as Part-of-Speech tags, dependency tags, or knowledge from medical dictionaries.
A Joint Training Approach to Tweet Classification and Adverse Effect Extraction and Normalization for SMM4H 2021
Authors: Mohab Elkaref, Lamiece Hassan
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Classification

In this work we describe our submissions to the Social Media Mining for Health (SMM4H) 2021 Shared Task. We investigated the effectiveness of a joint training approach to Task 1, specifically classification, extraction and normalization of Adverse Drug Effect (ADE) mentions in English tweets. Our approach performed well on the normalization task, achieving an above average f1 score of 24%, but less so on classification and extraction, with f1 scores of 22% and 37% respectively. Our experiments also showed that a larger dataset with more negative results led to stronger results than a smaller more balanced dataset, even when both datasets have the same positive examples. Finally we also submitted a tuned BERT model for Task 6: Classification of Covid-19 tweets containing symptoms, which achieved an above average f1 score of 96%.
Text Augmentation Techniques in Drug Adverse Effect Detection Task
Authors: Pavel Blinov
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Text Augmentation

The paper researches the problem of drug adverse effect detection in texts of social media. We describe the development of such classification system for Russian tweets. To increase the train dataset we apply a couple of augmentation techniques and analyze their effect in comparison with similar systems presented at 2021 years’ SMM4H Workshop.
Classification of Tweets Self-reporting Adverse Pregnancy Outcomes and Potential COVID-19 Cases Using RoBERTa Transformers
Authors: Lung-Hao Lee, Man-Chen Hung, Chien-Huan Lu, Chang-Hao Chen, Po-Lei Lee, Kuo-Kai Shyu
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Language Modeling, Language Modelling

This study describes our proposed model design for SMM4H 2021 shared tasks. We fine-tune the language model of RoBERTa transformers and their connecting classifier to complete the classification tasks of tweets for adverse pregnancy outcomes (Task 4) and potential COVID-19 cases (Task 5). The evaluation metric is F1-score of the positive class for both tasks. For Task 4, our best score of 0.93 exceeded the mean score of 0.925. For Task 5, our best of 0.75 exceeded the mean score of 0.745.
NLP@NISER: Classification of COVID19 tweets containing symptoms
Authors: Deepak Kumar, Nalin Kumar, Subhankar Mishra
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Classification, text-classification, Text Classification

In this paper, we describe our approaches for task six of Social Media Mining for Health Applications (SMM4H) shared task in 2021. The task is to classify twitter tweets containing COVID-19 symptoms in three classes (self-reports, non-personal reports & literature/news mentions). We implemented BERT and XLNet for this text classification task. Best result was achieved by XLNet approach, which is F1 score 0.94, precision 0.9448 and recall 0.94448. This is slightly better than the average score, i.e. F1 score 0.93, precision 0.93235 and recall 0.93235.
Identification of profession & occupation in Health-related Social Media using tweets in Spanish
Authors: Victoria Pachón, Jacinto Mata Vázquez, Juan Luís Domínguez Olmedo
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6

In this paper we present our approach and system description on Task 7a in ProfNer-ST: Identification of profession & occupation in Health related Social Media. Our main contribution is to show the effectiveness of using BETO-Spanish BERT as a model based on transformers pretrained with a Spanish Corpus for classification tasks. In our experiments we compared several architectures based on transformers with others based on classical machine learning algorithms. With this approach, we achieved an F1-score of 0.92 in the evaluation process.
Lasige-BioTM at ProfNER: BiLSTM-CRF and contextual Spanish embeddings for Named Entity Recognition and Tweet Binary Classification
Authors: Pedro Ruas, Vitor Andrade, Francisco Couto
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Binary Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Word Embeddings

The paper describes the participation of the Lasige-BioTM team at sub-tracks A and B of ProfNER, which was based on: i) a BiLSTM-CRF model that leverages contextual and classical word embeddings to recognize and classify the mentions, and ii) on a rule-based module to classify tweets. In the Evaluation phase, our model achieved a F1-score of 0.917 (0,031 more than the median) in sub-track A and a F1-score of 0.727 (0,034 less than the median) in sub-track B.
Adversities are all you need: Classification of self-reported breast cancer posts on Twitter using Adversarial Fine-tuning
Authors: Adarsh Kumar, Ojasv Kamal, Susmita Mazumdar
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: All, Language Modeling, Language Modelling

In this paper, we describe our system entry for Shared Task 8 at SMM4H-2021, which is on automatic classification of self-reported breast cancer posts on Twitter. In our system, we use a transformer-based language model fine-tuning approach to automatically identify tweets in the self-reports category. Furthermore, we involve a Gradient-based Adversarial fine-tuning to improve the overall model’s robustness. Our system achieved an F1-score of 0.8625 on the Development set and 0.8501 on the Test set in Shared Task-8 of SMM4H-2021.
UoB at ProfNER 2021: Data Augmentation for Classification Using Machine Translation
Authors: Frances Adriana Laureano De Leon, Harish Tayyar Madabushi, Mark Lee
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Data Augmentation, Machine Translation, Translation

This paper describes the participation of the UoB-NLP team in the ProfNER-ST shared subtask 7a. The task was aimed at detecting the mention of professions in social media text. Our team experimented with two methods of improving the performance of pre-trained models: Specifically, we experimented with data augmentation through translation and the merging of multiple language inputs to meet the objective of the task. While the best performing model on the test data consisted of mBERT fine-tuned on augmented data using back-translation, the improvement is minor possibly because multi-lingual pre-trained models such as mBERT already have access to the kind of information provided through back-translation and bilingual data.
IIITN NLP at SMM4H 2021 Tasks: Transformer Models for Classification on Health-Related Imbalanced Twitter Datasets
Authors: Varad Pimpalkhute, Prajwal Nakhate, Tausif Diwan
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Binary Classification, Data Augmentation

With increasing users sharing health-related information on social media, there has been a rise in using social media for health monitoring and surveillance. In this paper, we present a system that addresses classic health-related binary classification problems presented in Tasks 1a, 4, and 8 of the 6th edition of Social Media Mining for Health Applications (SMM4H) shared tasks. We developed a system based on RoBERTa (for Task 1a & 4) and BioBERT (for Task 8). Furthermore, we address the challenge of the imbalanced dataset and propose techniques such as undersampling, oversampling, and data augmentation to overcome the imbalanced nature of a given health-related dataset.
OCHADAI at SMM4H-2021 Task 5: Classifying self-reporting tweets on potential cases of COVID-19 by ensembling pre-trained language models
Authors: Ying Luo, Lis Pereira, Kobayashi Ichiro
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6

Since the outbreak of coronavirus at the end of 2019, there have been numerous studies on coro- navirus in the NLP arena. Meanwhile, Twitter has been a valuable source of news and a pub- lic medium for the conveyance of information and personal expression. This paper describes the system developed by the Ochadai team for the Social Media Mining for Health Appli- cations (SMM4H) 2021 Task 5, which aims to automatically distinguish English tweets that self-report potential cases of COVID-19 from those that do not. We proposed a model ensemble that leverages pre-trained represen- tations from COVID-Twitter-BERT (Müller et al., 2020), RoBERTa (Liu et al., 2019), and Twitter-RoBERTa (Glazkova et al., 2021). Our model obtained F1-scores of 76% on the test set in the evaluation phase, and 77.5% in the post-evaluation phase.
PAII-NLP at SMM4H 2021: Joint Extraction and Normalization of Adverse Drug Effect Mentions in Tweets
Authors: Zongcheng Ji, Tian Xia, Mei Han
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6

This paper describes our system developed for the subtask 1c of the sixth Social Media Mining for Health Applications (SMM4H) shared task in 2021. The aim of the subtask is to recognize the adverse drug effect (ADE) mentions from tweets and normalize the identified mentions to their mapping MedDRA preferred term IDs. Our system is based on a neural transition-based joint model, which is to perform recognition and normalization simultaneously. Our final two submissions outperform the average F1 score by 1-2%.
Assessing multiple word embeddings for named entity recognition of professions and occupations in health-related social media
Authors: Vasile Pais, Maria Mitrofan
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Word Embeddings

This paper presents our contribution to the ProfNER shared task. Our work focused on evaluating different pre-trained word embedding representations suitable for the task. We further explored combinations of embeddings in order to improve the overall results.
Classification of COVID19 tweets using Machine Learning Approaches
Authors: Anupam Mondal, Sainik Mahata, Monalisa Dey, Dipankar Das
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: BIG-bench Machine Learning, Classification

The reported work is a description of our participation in the “Classification of COVID19 tweets containing symptoms” shared task, organized by the “Social Media Mining for Health Applications (SMM4H)” workshop. The literature describes two machine learning approaches that were used to build a three class classification system, that categorizes tweets related to COVID19, into three classes, viz., self-reports, non-personal reports, and literature/news mentions. The steps for pre-processing tweets, feature extraction, and the development of the machine learning models, are described extensively in the documentation. Both the developed learning models, when evaluated by the organizers, garnered F1 scores of 0.93 and 0.92 respectively.
Fine-tuning BERT to classify COVID19 tweets containing symptoms
Authors: Rajarshi Roychoudhury, Sudip Naskar
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Articles, Word Embeddings

Twitter is a valuable source of patient-generated data that has been used in various population health studies. The first step in many of these studies is to identify and capture Twitter messages (tweets) containing medication mentions. Identifying personal mentions of COVID19 symptoms requires distinguishing personal mentions from other mentions such as symptoms reported by others and references to news articles or other sources. In this article, we describe our submission to Task 6 of the Social Media Mining for Health Applications (SMM4H) Shared Task 2021. This task challenged participants to classify tweets where the target classes are:(1) self-reports,(2) non-personal reports, and (3) literature/news mentions. Our system used a handcrafted preprocessing and word embeddings from BERT encoder model. We achieved an F1 score of 93%
Identifying professions & occupations in Health-related Social Media using Natural Language Processing
Authors: Alberto Mesa Murgado, Ana Parras Portillo, Pilar López Úbeda, Maite Martin, Alfonso Ureña-López
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Binary Classification, Classification, NER

This paper describes the entry of the research group SINAI at SMM4H’s ProfNER task on the identification of professions and occupations in social media related with health. Specifically we have participated in Task 7a: Tweet Binary Classification to determine whether a tweet contains mentions of occupations or not, as well as in Task 7b: NER Offset Detection and Classification aimed at predicting occupations mentions and classify them discriminating by professions and working statuses.
Approaching SMM4H with auto-regressive language models and back-translation
Authors: Joseph Cornelius, Tilia Ellendorff, Fabio Rinaldi
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: Translation

We describe our submissions to the 6th edition of the Social Media Mining for Health Applications (SMM4H) shared task. Our team (OGNLP) participated in the sub-task: Classification of tweets self-reporting potential cases of COVID-19 (Task 5). For our submissions, we employed systems based on auto-regressive transformer models (XLNet) and back-translation for balancing the dataset.
ULD-NUIG at Social Media Mining for Health Applications (#SMM4H) Shared Task 2021
Authors: Atul Kr. Ojha, Priya Rani, Koustava Goswami, Bharathi Raja Chakravarthi, John P. McCrae
Date: 2021-06-01 00:00:00
Proceeding: NAACL (SMM4H) 2021 6
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Social media platforms such as Twitter and Facebook have been utilised for various research studies, from the cohort-level discussion to community-driven approaches to address the challenges in utilizing social media data for health, clinical and biomedical information. Detection of medical jargon’s, named entity recognition, multi-word expression becomes the primary, fundamental steps in solving those challenges. In this paper, we enumerate the ULD-NUIG team’s system, designed as part of Social Media Mining for Health Applications (#SMM4H) Shared Task 2021. The team conducted a series of experiments to explore the challenges of task 6 and task 5. The submitted systems achieve F-1 0.84 and 0.53 score for task 6 and 5 respectively.
Knowing Right from Wrong: Should We Use More Complex Models for Automatic Short-Answer Scoring in Bahasa Indonesia?
Authors: Ali Akbar Septiandri, Yosef Ardhito Winatmoko, Ilham Firdausi Putra
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (sustainlp) 2020 11
Tasks: BIG-bench Machine Learning, Deep Learning

We compare three solutions to UKARA 1.0 challenge on automated short-answer scoring: single classical, ensemble classical, and deep learning. The task is to classify given answers to two questions, whether they are right or wrong. While recent development shows increasing model complexity to push the benchmark performances, they tend to be resource-demanding with mundane improvement. For the UKARA task, we found that bag-of-words and classical machine learning approaches can compete with ensemble models and Bi-LSTM model with pre-trained word2vec embedding from 200 million words. In this case, the single classical machine learning achieved less than 2% difference in F1 compared to the deep learning approach with 1/18 time for model training.