pwc-archive/papers-with-abstracts huggingface dataset:
NLM at MEDIQA 2021: Transfer Learning-based Approaches for Consumer Question and Multi-Answer Summarization
Authors: Shweta Yadav, Mourad Sarrouti, Deepak Gupta
Date: 2021-06-01 00:00:00
Proceeding: NAACL (BioNLP) 2021 6
Tasks: Descriptive, Question Answering, Retrieval, Transfer Learning

The quest for seeking health information has swamped the web with consumers’ healthrelated questions, which makes the need for efficient and reliable question answering systems more pressing. The consumers’ questions, however, are very descriptive and contain several peripheral information (like patient’s medical history, demographic information, etc.), that are often not required for answering the question. Furthermore, it contributes to the challenges of understanding natural language questions for automatic answer retrieval. Also, it is crucial to provide the consumers with the exact and relevant answers, rather than the entire pool of answer documents to their question. One of the cardinal tasks in achieving robust consumer health question answering systems is the question summarization and multi-document answer summarization. This paper describes the participation of the U.S. National Library of Medicine (NLM) in Consumer Question and Multi-Answer Summarization tasks of the MEDIQA 2021 challenge at NAACL-BioNLP workshop. In this work, we exploited the capabilities of pre-trained transformer models and introduced a transfer learning approach for the abstractive Question Summarization and extractive Multi-Answer Summarization tasks by first pre-training our model on a task-specific summarization dataset followed by fine-tuning it for both the tasks via incorporating medical entities. We achieved the second, sixth and the fourth position for the Question Summarization task in terms ROUGE-1, ROUGE-2 and ROUGE-L scores respectively.
IBMResearch at MEDIQA 2021: Toward Improving Factual Correctness of Radiology Report Abstractive Summarization
Authors: Diwakar Mahajan, Ching-Huei Tsou, Jennifer J Liang
Date: 2021-06-01 00:00:00
Proceeding: NAACL (BioNLP) 2021 6
Tasks: Abstractive Text Summarization, Decoder, Re-Ranking

Although recent advances in abstractive summarization systems have achieved high scores on standard natural language metrics like ROUGE, their lack of factual consistency remains an open challenge for their use in sensitive real-world settings such as clinical practice. In this work, we propose a novel approach to improve factual correctness of a summarization system by re-ranking the candidate summaries based on a factual vector of the summary. We applied this process during our participation in MEDIQA 2021 Task 3: Radiology Report Summarization, where the task is to generate an impression summary of a radiology report, given findings and background as inputs. In our system, we first used a transformer-based encoder-decoder model to generate top N candidate impression summaries for a report, then trained another transformer-based model to predict a 14-observations-vector of the impression based on the findings and background of the report, and finally, utilized this vector to re-rank the candidate summaries. We also employed a source-specific ensembling technique to accommodate for distinct writing styles from different radiology report sources. Our approach yielded 2nd place in the challenge.
UETrice at MEDIQA 2021: A Prosper-thy-neighbour Extractive Multi-document Summarization Model
Authors: Duy-Cat Can, Quoc-An Nguyen, Quoc-Hung Duong, Minh-Quang Nguyen, Huy-Son Nguyen, Linh Nguyen Tran Ngoc, Quang-Thuy Ha, Mai-Vu Tran
Date: 2021-06-01 00:00:00
Proceeding: NAACL (BioNLP) 2021 6
Tasks: Document Summarization, Extractive Summarization, Multi-Document Summarization

This paper describes a system developed to summarize multiple answers challenge in the MEDIQA 2021 shared task collocated with the BioNLP 2021 Workshop. We propose an extractive summarization architecture based on several scores and state-of-the-art techniques. We also present our novel prosper-thy-neighbour strategies to improve performance. Our model has been proven to be effective with the best ROUGE-1/ROUGE-L scores, being the shared task runner up by ROUGE-2 F1 score (over 13 participated teams).
MNLP at MEDIQA 2021: Fine-Tuning PEGASUS for Consumer Health Question Summarization
Authors: Jooyeon Lee, Huong Dang, Ozlem Uzuner, Sam Henry
Date: 2021-06-01 00:00:00
Proceeding: NAACL (BioNLP) 2021 6
Tasks: Abstractive Text Summarization, Question Answering

This paper details a Consumer Health Question (CHQ) summarization model submitted to MEDIQA 2021 for shared task 1: Question Summarization. Many CHQs are composed of multiple sentences with typos or unnecessary information, which can interfere with automated question answering systems. Question summarization mitigates this issue by removing this unnecessary information, aiding automated systems in generating a more accurate summary. Our summarization approach focuses on applying multiple pre-processing techniques, including question focus identification on the input and the development of an ensemble method to combine question focus with an abstractive summarization method. We use the state-of-art abstractive summarization model, PEGASUS (Pre-training with Extracted Gap-sentences for Abstractive Summarization), to generate abstractive summaries. Our experiments show that using our ensemble method, which combines abstractive summarization with question focus identification, improves performance over using summarization alone. Our model shows a ROUGE-2 F-measure of 11.14% against the official test dataset.
UETfishes at MEDIQA 2021: Standing-on-the-Shoulders-of-Giants Model for Abstractive Multi-answer Summarization
Authors: Hoang-Quynh Le, Quoc-An Nguyen, Quoc-Hung Duong, Minh-Quang Nguyen, Huy-Son Nguyen, Tam Doan Thanh, Hai-Yen Thi Vuong, Trang M. Nguyen
Date: 2021-06-01 00:00:00
Proceeding: NAACL (BioNLP) 2021 6
Tasks: Abstractive Text Summarization, Denoising

This paper describes a system developed to summarize multiple answers challenge in the MEDIQA 2021 shared task collocated with the BioNLP 2021 Workshop. We present an abstractive summarization model based on BART, a denoising auto-encoder for pre-training sequence-to-sequence models. As focusing on the summarization of answers to consumer health questions, we propose a query-driven filtering phase to choose useful information from the input document automatically. Our approach achieves potential results, rank no.2 (evaluated on extractive references) and no.3 (evaluated on abstractive references) in the final evaluation.
Language that Captivates the Audience: Predicting Affective Ratings of TED Talks in a Multi-Label Classification Task
Authors: Elma Kerz, Yu Qiao, Daniel Wiechmann
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

The aim of the paper is twofold: (1) to automatically predict the ratings assigned by viewers to 14 categories available for TED talks in a multi-label classification task and (2) to determine what types of features drive classification accuracy for each of the categories. The focus is on features of language usage from five groups pertaining to syntactic complexity, lexical richness, register-based n-gram measures, information-theoretic measures and LIWC-style measures. We show that a Recurrent Neural Network classifier trained exclusively on within-text distributions of such features can reach relatively high levels of overall accuracy (69%) across the 14 categories. We find that features from two groups are strong predictors of the affective ratings across all categories and that there are distinct patterns of language usage for each rating category.
Partisanship and Fear are Associated with Resistance to COVID-19 Directives
Authors: Mike Lindow, David DeFranza, Arul Mishra, Himanshu Mishra
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4

Ideological differences have had a large impact on individual and community response to the COVID-19 pandemic in the United States. Early behavioral research during the pandemic showed that conservatives were less likely to adhere to health directives, which contradicts a body of work suggesting that conservative ideology emphasizes a rule abiding, loss aversion, and prevention focus. We reconcile this contradiction by analyzing semantic content of local press releases, federal press releases, and localized tweets during the first month of the government response to COVID-19 in the United States. Controlling for factors such as COVID-19 confirmed cases and deaths, local economic indicators, and more, we find that online expressions of fear in conservative areas lead to an increase in adherence to public health recommendations concerning COVID-19, and that expressions of fear in government press releases are a significant predictor of expressed fear on Twitter.
Explainable Detection of Sarcasm in Social Media
Authors: Ramya Akula, Ivan Garibay
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Sarcasm Detection

Sarcasm is a linguistic expression often used to communicate the opposite of what is said, usually something that is very unpleasant with an intention to insult or ridicule. Inherent ambiguity in sarcastic expressions makes sarcasm detection very difficult. In this work, we focus on detecting sarcasm in textual conversations, written in English, from various social networking platforms and online media. To this end, we develop an interpretable deep learning model using multi-head self-attention and gated recurrent units. We show the effectiveness and interpretability of our approach by achieving state-of-the-art results on datasets from social networking platforms, online discussion forums, and political dialogues.
Disentangling Document Topic and Author Gender in Multiple Languages: Lessons for Adversarial Debiasing
Authors: Erenay Dayanik, Sebastian Padó
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Classification, text-classification, Text Classification

Text classification is a central tool in NLP. However, when the target classes are strongly correlated with other textual attributes, text classification models can pick up “wrong” features, leading to bad generalization and biases. In social media analysis, this problem surfaces for demographic user classes such as language, topic, or gender, which influence the generate text to a substantial extent. Adversarial training has been claimed to mitigate this problem, but thorough evaluation is missing. In this paper, we experiment with text classification of the correlated attributes of document topic and author gender, using a novel multilingual parallel corpus of TED talk transcripts. Our findings are: (a) individual classifiers for topic and author gender are indeed biased; (b) debiasing with adversarial training works for topic, but breaks down for author gender; (c) gender debiasing results differ across languages. We interpret the result in terms of feature space overlap, highlighting the role of linguistic surface realization of the target classes.
Universal Joy A Data Set and Results for Classifying Emotions Across Languages
Authors: Sotiris Lamprinidis, Federico Bianchi, Daniel Hardt, Dirk Hovy
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Diversity, Emotion Classification, Zero-Shot Learning

While emotions are universal aspects of human psychology, they are expressed differently across different languages and cultures. We introduce a new data set of over 530k anonymized public Facebook posts across 18 languages, labeled with five different emotions. Using multilingual BERT embeddings, we show that emotions can be reliably inferred both within and across languages. Zero-shot learning produces promising results for low-resource languages. Following established theories of basic emotions, we provide a detailed analysis of the possibilities and limits of cross-lingual emotion classification. We find that structural and typological similarity between languages facilitates cross-lingual learning, as well as linguistic diversity of training data. Our results suggest that there are commonalities underlying the expression of emotion in different languages. We publicly release the anonymized data for future research.
FEEL-IT: Emotion and Sentiment Classification for the Italian Language
Authors: Federico Bianchi, Debora Nozza, Dirk Hovy
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Classification, Sentiment Analysis, Sentiment Classification

While sentiment analysis is a popular task to understand people’s reactions online, we often need more nuanced information: is the post negative because the user is angry or sad? An abundance of approaches have been introduced for tackling these tasks, also for Italian, but they all treat only one of the tasks. We introduce FEEL-IT, a novel benchmark corpus of Italian Twitter posts annotated with four basic emotions: anger, fear, joy, sadness. By collapsing them, we can also do sentiment analysis. We evaluate our corpus on benchmark datasets for both emotion and sentiment classification, obtaining competitive results. We release an open-source Python library, so researchers can use a model trained on FEEL-IT for inferring both sentiments and emotions from Italian text.
WASSA 2021 Shared Task: Predicting Empathy and Emotion in Reaction to News Stories
Authors: Shabnam Tafreshi, Orphee De Clercq, Valentin Barriere, Sven Buechel, João Sedoc, Alexandra Balahur
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Articles, Sentence

This paper presents the results that were obtained from the WASSA 2021 shared task on predicting empathy and emotions. The participants were given access to a dataset comprising empathic reactions to news stories where harm is done to a person, group, or other. These reactions consist of essays, Batson empathic concern, and personal distress scores, and the dataset was further extended with news articles, person-level demographic information (age, gender, ethnicity, income, education level), and personality information. Additionally, emotion labels, namely Ekman’s six basic emotions, were added to the essays at both the document and sentence level. Participation was encouraged in two tracks: predicting empathy and predicting emotion categories. In total five teams participated in the shared task. We summarize the methods and resources used by the participating teams.
Lightweight Models for Multimodal Sequential Data
Authors: Soumya Sourav, Jessica Ouyang
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Multimodal Sentiment Analysis, Sentiment Analysis

Human language encompasses more than just text; it also conveys emotions through tone and gestures. We present a case study of three simple and efficient Transformer-based architectures for predicting sentiment and emotion in multimodal data. The Late Fusion model merges unimodal features to create a multimodal feature sequence, the Round Robin model iteratively combines bimodal features using cross-modal attention, and the Hybrid Fusion model combines trimodal and unimodal features together to form a final feature sequence for predicting sentiment. Our experiments show that our small models are effective and outperform the publicly released versions of much larger, state-of-the-art multimodal sentiment analysis systems.
Exploring Implicit Sentiment Evoked by Fine-grained News Events
Authors: Cynthia Van Hee, Orphee De Clercq, Veronique Hoste
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Articles, Collaborative Filtering, Diversity, Sentiment Analysis

We investigate the feasibility of defining sentiment evoked by fine-grained news events. Our research question is based on the premise that methods for detecting implicit sentiment in news can be a key driver of content diversity, which is one way to mitigate the detrimental effects of filter bubbles that recommenders based on collaborative filtering may produce. Our experiments are based on 1,735 news articles from major Flemish newspapers that were manually annotated, with high agreement, for implicit sentiment. While lexical resources prove insufficient for sentiment analysis in this data genre, our results demonstrate that machine learning models based on SVM and BERT are able to automatically infer the implicit sentiment evoked by news events.
Exploring Stylometric and Emotion-Based Features for Multilingual Cross-Domain Hate Speech Detection
Authors: Ilia Markov, Nikola Ljubešić, Darja Fišer, Walter Daelemans
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Hate Speech Detection

In this paper, we describe experiments designed to evaluate the impact of stylometric and emotion-based features on hate speech detection: the task of classifying textual content into hate or non-hate speech classes. Our experiments are conducted for three languages – English, Slovene, and Dutch – both in in-domain and cross-domain setups, and aim to investigate hate speech using features that model two linguistic phenomena: the writing style of hateful social media content operationalized as function word usage on the one hand, and emotion expression in hateful messages on the other hand. The results of experiments with features that model different combinations of these phenomena support our hypothesis that stylometric and emotion-based features are robust indicators of hate speech. Their contribution remains persistent with respect to domain and language variation. We show that the combination of features that model the targeted phenomena outperforms words and character n-gram features under cross-domain conditions, and provides a significant boost to deep learning models, which currently obtain the best results, when combined with them in an ensemble.
Synthetic Examples Improve Cross-Target Generalization: A Study on Stance Detection on a Twitter corpus.
Authors: Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, Nigel Collier
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Stance Detection

Cross-target generalization is a known problem in stance detection (SD), where systems tend to perform poorly when exposed to targets unseen during training. Given that data annotation is expensive and time-consuming, finding ways to leverage abundant unlabeled in-domain data can offer great benefits. In this paper, we apply a weakly supervised framework to enhance cross-target generalization through synthetically annotated data. We focus on Twitter SD and show experimentally that integrating synthetic data is helpful for cross-target generalization, leading to significant improvements in performance, with gains in F1 scores ranging from +3.4 to +5.1.
Creating and Evaluating Resources for Sentiment Analysis in the Low-resource Language: Sindhi
Authors: Wazir Ali, Naveed Ali, Yong Dai, Jay Kumar, Saifullah Tumrani, Zenglin Xu
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Sentiment Analysis, Subjectivity Analysis, text annotation

In this paper, we develop Sindhi subjective lexicon using a merger of existing English resources: NRC lexicon, list of opinion words, SentiWordNet, Sindhi-English bilingual dictionary, and collection of Sindhi modifiers. The positive or negative sentiment score is assigned to each Sindhi opinion word. Afterwards, we determine the coverage of the proposed lexicon with subjectivity analysis. Moreover, we crawl multi-domain tweet corpus of news, sports, and finance. The crawled corpus is annotated by experienced annotators using the Doccano text annotation tool. The sentiment annotated corpus is evaluated by employing support vector machine (SVM), recurrent neural network (RNN) variants, and convolutional neural network (CNN).
Multi-Emotion Classification for Song Lyrics
Authors: Darren Edmonds, João Sedoc
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Classification, Emotion Classification

Song lyrics convey a multitude of emotions to the listener and powerfully portray the emotional state of the writer or singer. This paper examines a variety of modeling approaches to the multi-emotion classification problem for songs. We introduce the Edmonds Dance dataset, a novel emotion-annotated lyrics dataset from the reader’s perspective, and annotate the dataset of Mihalcea and Strapparava (2012) at the song level. We find that models trained on relatively small song datasets achieve marginally better performance than BERT (Devlin et al., 2018) fine-tuned on large social media or dialog datasets.
ONE: Toward ONE model, ONE algorithm, ONE corpus dedicated to sentiment analysis of Arabic/Arabizi and its dialects
Authors: Imane Guellil, Faical Azouaou, Fodil Benali, Hachani Ala-Eddine
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Language Modeling, Language Modelling, Sentiment Analysis

Arabic is the official language of 22 countries, spoken by more than 400 million speakers. Each one of this country use at least on dialect for daily life conversation. Then, Arabic has at least 22 dialects. Each dialect can be written in Arabic or Arabizi Scripts. The most recent researches focus on constructing a language model and a training corpus for each dialect, in each script. Following this technique means constructing 46 different resources (by including the Modern Standard Arabic, MSA) for handling only one language. In this paper, we extract ONE corpus, and we propose ONE algorithm to automatically construct ONE training corpus using ONE classification model architecture for sentiment analysis MSA and different dialects. After manually reviewing the training corpus, the obtained results outperform all the research literature results for the targeted test corpora.
Me, myself, and ire: Effects of automatic transcription quality on emotion, sarcasm, and personality detection
Authors: John Culnan, SeongJin Park, Meghavarshini Krishnaswamy, Rebecca Sharp
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4

In deployment, systems that use speech as input must make use of automated transcriptions. Yet, typically when these systems are evaluated, gold transcriptions are assumed. We explicitly examine the impact of transcription errors on the downstream performance of a multi-modal system on three related tasks from three datasets: emotion, sarcasm, and personality detection. We include three separate transcription tools and show that while all automated transcriptions propagate errors that substantially impact downstream performance, the open-source tools fair worse than the paid tool, though not always straightforwardly, and word error rates do not correlate well with downstream performance. We further find that the inclusion of audio features partially mitigates transcription errors, but that a naive usage of a multi-task setup does not.
Emotional RobBERT and Insensitive BERTje: Combining Transformers and Affect Lexica for Dutch Emotion Detection
Authors: Luna De Bruyne, Orphee De Clercq, Veronique Hoste
Date: 2021-04-01 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Language Modeling, Language Modelling, Meta-Learning

In a first step towards improving Dutch emotion detection, we try to combine the Dutch transformer models BERTje and RobBERT with lexicon-based methods. We propose two architectures: one in which lexicon information is directly injected into the transformer model and a meta-learning approach where predictions from transformers are combined with lexicon features. The models are tested on 1,000 Dutch tweets and 1,000 captions from TV-shows which have been manually annotated with emotion categories and dimensions. We find that RobBERT clearly outperforms BERTje, but that directly adding lexicon information to transformers does not improve performance. In the meta-learning approach, lexicon information does have a positive effect on BERTje, but not on RobBERT. This suggests that more emotional information is already contained within this latter language model.