pwc-archive/papers-with-abstracts huggingface dataset:
VoiSeR: A New Benchmark for Voice-Based Search Refinement
Authors: Simone Filice, Giuseppe Castellucci, Marcus Collins, Eugene Agichtein, Oleg Rokhlenko
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Attribute, Conversational Search

Voice assistants, e.g., Alexa or Google Assistant, have dramatically improved in recent years. Supporting voice-based search, exploration, and refinement are fundamental tasks for voice assistants, and remain an open challenge. For example, when using voice to search an online shopping site, a user often needs to refine their search by some aspect or facet. This common user intent is usually available through a {``}filter-by{''} interface on online shopping websites, but is challenging to support naturally via voice, as the intent of refinements must be interpreted in the context of the original search, the initial results, and the available product catalogue facets. To our knowledge, no benchmark dataset exists for training or validating such contextual search understanding models. To bridge this gap, we introduce the first large-scale dataset of voice-based search refinements, VoiSeR, consisting of about 10,000 search refinement utterances, collected using a novel crowdsourcing task. These utterances are intended to refine a previous search, with respect to a search facet or attribute (e.g., brand, color, review rating, etc.), and are manually annotated with the specific intent. This paper reports qualitative and empirical insights into the most common and challenging types of refinements that a voice-based conversational search system must support. As we show, VoiSeR can support research in conversational query understanding, contextual user intent prediction, and other conversational search topics to facilitate the development of conversational search systems.
Lexical Normalization for Code-switched Data and its Effect on POS Tagging
Authors: Rob van der Goot, {\"O}zlem {\c{C}}etino{\u{g}}lu
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Lexical Normalization, POS, POS Tagging, Translation

Lexical normalization, the translation of non-canonical data to standard language, has shown to improve the performance of many natural language processing tasks on social media. Yet, using multiple languages in one utterance, also called code-switching (CS), is frequently overlooked by these normalization systems, despite its common use in social media. In this paper, we propose three normalization models specifically designed to handle code-switched data which we evaluate for two language pairs: Indonesian-English and Turkish-German. For the latter, we introduce novel normalization layers and their corresponding language ID and POS tags for the dataset, and evaluate the downstream effect of normalization on POS tagging. Results show that our CS-tailored normalization models significantly outperform monolingual ones, and lead to 5.4{\%} relative performance increase for POS tagging as compared to unnormalized input.
Structural Encoding and Pre-training Matter: Adapting BERT for Table-Based Fact Verification
Authors: Rui Dong, David Smith
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Fact Verification, Graph Neural Network, Language Modeling, Language Modelling, Misinformation, Question Answering, Table-based Fact Verification, Table-to-Text Generation, Text Generation

Growing concern with online misinformation has encouraged NLP research on fact verification. Since writers often base their assertions on structured data, we focus here on verifying textual statements given evidence in tables. Starting from the Table Parsing (TAPAS) model developed for question answering (Herzig et al., 2020), we find that modeling table structure improves a language model pre-trained on unstructured text. Pre-training language models on English Wikipedia table data further improves performance. Pre-training on a question answering task with column-level cell rank information achieves the best performance. With improved pre-training and cell embeddings, this approach outperforms the state-of-the-art Numerically-aware Graph Neural Network table fact verification model (GNN-TabFact), increasing statement classification accuracy from 72.2{\%} to 73.9{\%} even without modeling numerical information. Incorporating numerical information with cell rankings and pre-training on a question-answering task increases accuracy to 76{\%}. We further analyze accuracy on statements implicating single rows or multiple rows and columns of tables, on different numerical reasoning subtasks, and on generalizing to detecting errors in statements derived from the ToTTo table-to-text generation dataset.
PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media
Authors: Ramit Sawhney, Harshit Joshi, Lucie Flek, Rajiv Ratn Shah
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2

Recent psychological studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners. Contextualizing the build-up of such ideation is critical for the identification of users at risk. In this work, we focus on identifying suicidal intent in tweets by augmenting linguistic models with emotional phases modeled from users{'} historical context. We propose PHASE, a time-and phase-aware framework that adaptively learns features from a user{'}s historical emotional spectrum on Twitter for preliminary screening of suicidal risk. Building on clinical studies, PHASE learns phase-like progressions in users{'} historical Plutchik-wheel-based emotions to contextualize suicidal intent. While outperforming state-of-the-art methods, we show the utility of temporal and phase-based emotional contextual cues for suicide ideation detection. We further discuss practical and ethical considerations.
Exploiting Definitions for Frame Identification
Authors: Tianyu Jiang, Ellen Riloff
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Semantic Parsing, Sentence

Frame identification is one of the key challenges for frame-semantic parsing. The goal of this task is to determine which frame best captures the meaning of a target word or phrase in a sentence. We present a new model for frame identification that uses a pre-trained transformer model to generate representations for frames and lexical units (senses) using their formal definitions in FrameNet. Our frame identification model assesses the suitability of a frame for a target word in a sentence based on the semantic coherence of their meanings. We evaluate our model on three data sets and show that it consistently achieves better performance than previous systems.
Conceptual Grounding Constraints for Truly Robust Biomedical Name Representations
Authors: Pieter Fivez, Simon Suster, Walter Daelemans
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2

Effective representation of biomedical names for downstream NLP tasks requires the encoding of both lexical as well as domain-specific semantic information. Ideally, the synonymy and semantic relatedness of names should be consistently reflected by their closeness in an embedding space. To achieve such robustness, prior research has considered multi-task objectives when training neural encoders. In this paper, we take a next step towards truly robust representations, which capture more domain-specific semantics while remaining universally applicable across different biomedical corpora and domains. To this end, we use conceptual grounding constraints which more effectively align encoded names to pretrained embeddings of their concept identifiers. These constraints are effective even when using a Deep Averaging Network, a simple feedforward encoding architecture that allows for scaling to large corpora while remaining sufficiently expressive. We empirically validate our approach using multiple tasks and benchmarks, which assess both literal synonymy as well as more general semantic relatedness.
Adaptive Mixed Component LDA for Low Resource Topic Modeling
Authors: Suzanna Sia, Kevin Duh
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Topic Models

Probabilistic topic models in low data resource scenarios are faced with less reliable estimates due to sparsity of discrete word co-occurrence counts, and do not have the luxury of retraining word or topic embeddings using neural methods. In this challenging resource constrained setting, we explore mixture models which interpolate between the discrete and continuous topic-word distributions that utilise pre-trained embeddings to improve topic coherence. We introduce an automatic trade-off between the discrete and continuous representations via an adaptive mixture coefficient, which places greater weight on the discrete representation when the corpus statistics are more reliable. The adaptive mixture coefficient takes into account global corpus statistics, and the uncertainty in each topic{'}s continuous distributions. Our approach outperforms the fully discrete, fully continuous, and static mixture model on topic coherence in low resource settings. We additionally demonstrate the generalisability of our method by extending it to handle multilingual document collections.
Evaluating Neural Model Robustness for Machine Comprehension
Authors: Winston Wu, Dustin Arendt, Svitlana Volkova
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Adversarial Attack, model, Reading Comprehension, Sentence, TriviaQA

We evaluate neural model robustness to adversarial attacks using different types of linguistic unit perturbations {--} character and word, and propose a new method for strategic sentence-level perturbations. We experiment with different amounts of perturbations to examine model confidence and misclassification rate, and contrast model performance with different embeddings BERT and ELMo on two benchmark datasets SQuAD and TriviaQA. We demonstrate how to improve model performance during an adversarial attack by using ensembles. Finally, we analyze factors that effect model behavior under adversarial attack, and develop a new model to predict errors during attacks. Our novel findings reveal that (a) unlike BERT, models that use ELMo embeddings are more susceptible to adversarial attacks, (b) unlike word and paraphrase, character perturbations affect the model the most but are most easily compensated for by adversarial training, (c) word perturbations lead to more high-confidence misclassifications compared to sentence- and character-level perturbations, (d) the type of question and model answer length (the longer the answer the more likely it is to be incorrect) is the most predictive of model errors in adversarial setting, and (e) conclusions about model behavior are dataset-specific.
Context-aware Neural Machine Translation with Mini-batch Embedding
Authors: Makoto Morishita, Jun Suzuki, Tomoharu Iwata, Masaaki Nagata
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Machine Translation, NMT, Sentence, Translation

It is crucial to provide an inter-sentence context in Neural Machine Translation (NMT) models for higher-quality translation. With the aim of using a simple approach to incorporate inter-sentence information, we propose mini-batch embedding (MBE) as a way to represent the features of sentences in a mini-batch. We construct a mini-batch by choosing sentences from the same document, and thus the MBE is expected to have contextual information across sentences. Here, we incorporate MBE in an NMT model, and our experiments show that the proposed method consistently outperforms the translation capabilities of strong baselines and improves writing style or terminology to fit the document{'}s context.
Project-then-Transfer: Effective Two-stage Cross-lingual Transfer for Semantic Dependency Parsing
Authors: Hiroaki Ozaki, Gaku Morio, Terufumi Morishita, Toshinori Miyoshi
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Cross-Lingual Transfer, Dependency Parsing, Semantic Dependency Parsing

This paper describes the first report on cross-lingual transfer for semantic dependency parsing. We present the insight that there are twodifferent kinds of cross-linguality, namely sur-face level and mantic level, and try to cap-ture both kinds of cross-linguality by combin-ing annotation projection and model transferof pre-trained language models. Our exper-iments showed that the performance of our graph-based semantic dependency parser almost achieved the approximated upper bound.
On the Computational Modelling of Michif Verbal Morphology
Authors: Fineen Davis, Eddie Antonio Santos, Heather Souter
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Language Modeling, Language Modelling

This paper presents a finite-state computational model of the verbal morphology of Michif. Michif, the official language of the M{\'e}tis peoples, is a uniquely mixed language with Algonquian and French origins. It is spoken across the M{\'e}tis homelands in what is now called Canada and the United States, but it is highly endangered with less than 100 speakers. The verbal morphology is remarkably complex, as the already polysynthetic Algonquian patterns are combined with French elements and unique morpho-phonological interactions.The model presented in this paper, LI VERB KAA-OOSHITAHK DI MICHIF handles this complexity by using a series of composed finite-state transducers to model the concatenative morphology and phonological rule alternations that are unique to Michif. Such a rule-based approach is necessary as there is insufficient language data for an approach that uses machine learning. A language model such as LI VERB KAA-OOSHITAHK DI MICHIF furthers the goals of Indigenous computational linguistics in Canada while also supporting the creation of tools for documentation, education, and revitalization that are desired by the M{\'e}tis community.
A Few Topical Tweets are Enough for Effective User Stance Detection
Authors: Younes Samih, Kareem Darwish
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Clustering, Stance Detection

User stance detection entails ascertaining the position of a user towards a target, such as an entity, topic, or claim. Recent work that employs unsupervised classification has shown that performing stance detection on vocal Twitter users, who have many tweets on a target, can be highly accurate (+98{\%}). However, such methods perform poorly or fail completely for less vocal users, who may have authored only a few tweets about a target. In this paper, we tackle stance detection for such users using two approaches. In the first approach, we improve user-level stance detection by representing tweets using contextualized embeddings, which capture latent meanings of words in context. We show that this approach outperforms two strong baselines and achieves 89.6{\%} accuracy and 91.3{\%} macro F-measure on eight controversial topics. In the second approach, we expand the tweets of a given user using their Twitter timeline tweets, which may not be topically relevant, and then we perform unsupervised classification of the user, which entails clustering a user with other users in the training set. This approach achieves 95.6{\%} accuracy and 93.1{\%} macro F-measure.
How Good (really) are Grammatical Error Correction Systems?
Authors: Alla Rozovskaya, Dan Roth
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Grammatical Error Correction

Standard evaluations of Grammatical Error Correction (GEC) systems make use of a fixed reference text generated relative to the original text; they show, even when using multiple references, that we have a long way to go. This analysis paper studies the performance of GEC systems relative to closest-gold {--} a gold reference text created relative to the output of a system. Surprisingly, we show that the real performance is 20-40 points better than standard evaluations show. Moreover, the performance remains high even when considering any of the top-10 hypotheses produced by a system. Importantly, the type of mistakes corrected by lower-ranked hypotheses differs in interesting ways from the top one, providing an opportunity to focus on a range of errors {--} local spelling and grammar edits vs. more complex lexical improvements. Our study shows these results in English and Russian, and thus provides a preliminary proposal for a more realistic evaluation of GEC systems.
BERTective: Language Models and Contextual Information for Deception Detection
Authors: Tommaso Fornaciari, Federico Bianchi, Massimo Poesio, Dirk Hovy
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Deception Detection

Spotting a lie is challenging but has an enormous potential impact on security as well as private and public safety. Several NLP methods have been proposed to classify texts as truthful or deceptive. In most cases, however, the target texts{'} preceding context is not considered. This is a severe limitation, as any communication takes place in context, not in a vacuum, and context can help to detect deception. We study a corpus of Italian dialogues containing deceptive statements and implement deep neural models that incorporate various linguistic contexts. We establish a new state-of-the-art identifying deception and find that not all context is equally useful to the task. Only the texts closest to the target, if from the same speaker (rather than questions by an interlocutor), boost performance. We also find that the semantic information in language models such as BERT contributes to the performance. However, BERT alone does not capture the implicit knowledge of deception cues: its contribution is conditional on the concurrent use of attention to learn cues from BERT{'}s representations.
Fine-Grained Event Trigger Detection
Authors: Duong Le, Thien Huu Nguyen
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Event Detection, Word Sense Disambiguation

Most of the previous work on Event Detection (ED) has only considered the datasets with a small number of event types (i.e., up to 38 types). In this work, we present the first study on fine-grained ED (FED) where the evaluation dataset involves much more fine-grained event types (i.e., 449 types). We propose a novel method to transform the Semcor dataset for Word Sense Disambiguation into a large and high-quality dataset for FED. Extensive evaluation of the current ED methods is conducted to demonstrate the challenges of the generated datasets for FED, calling for more research effort in this area.
`Just because you are right, doesn't mean I am wrong': Overcoming a bottleneck in development and evaluation of Open-Ended VQA tasks
Authors: Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA), Visual Reasoning

GQA (CITATION) is a dataset for real-world visual reasoning and compositional question answering. We found that many answers predicted by the best vision-language models on the GQA dataset do not match the ground-truth answer but still are semantically meaningful and correct in the given context. In fact, this is the case with most existing visual question answering (VQA) datasets where they assume only one ground-truth answer for each question. We propose Alternative Answer Sets (AAS) of ground-truth answers to address this limitation, which is created automatically using off-the-shelf NLP tools. We introduce a semantic metric based on AAS and modify top VQA solvers to support multiple plausible answers for a question. We implement this approach on the GQA dataset and show the performance improvements.
Measuring and Improving Faithfulness of Attention in Neural Machine Translation
Authors: Pooya Moradi, Nishant Kambhatla, Anoop Sarkar
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Machine Translation, NMT, Translation

While the attention heatmaps produced by neural machine translation (NMT) models seem insightful, there is little evidence that they reflect a model{'}s true internal reasoning. We provide a measure of faithfulness for NMT based on a variety of stress tests where attention weights which are crucial for prediction are perturbed and the model should alter its predictions if the learned weights are a faithful explanation of the predictions. We show that our proposed faithfulness measure for NMT models can be improved using a novel differentiable objective that rewards faithful behaviour by the model through probability divergence. Our experimental results on multiple language pairs show that our objective function is effective in increasing faithfulness and can lead to a useful analysis of NMT model behaviour and more trustworthy attention heatmaps. Our proposed objective improves faithfulness without reducing the translation quality and has a useful regularization effect on the NMT model and can even improve translation quality in some cases.
ENPAR:Enhancing Entity and Entity Pair Representations for Joint Entity Relation Extraction
Authors: Yijun Wang, Changzhi Sun, Yuanbin Wu, Hao Zhou, Lei LI, Junchi Yan
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: coreference-resolution, Coreference Resolution, Entity Typing, Event Extraction, Multi-Task Learning, Relation, Relation Extraction, Sentence

Current state-of-the-art systems for joint entity relation extraction (Luan et al., 2019; Wad-den et al., 2019) usually adopt the multi-task learning framework. However, annotations for these additional tasks such as coreference resolution and event extraction are always equally hard (or even harder) to obtain. In this work, we propose a pre-training method ENPAR to improve the joint extraction performance. ENPAR requires only the additional entity annotations that are much easier to collect. Unlike most existing works that only consider incorporating entity information into the sentence encoder, we further utilize the entity pair information. Specifically, we devise four novel objectives,i.e., masked entity typing, masked entity prediction, adversarial context discrimination, and permutation prediction, to pre-train an entity encoder and an entity pair encoder. Comprehensive experiments show that the proposed pre-training method achieves significant improvement over BERT on ACE05, SciERC, and NYT, and outperforms current state-of-the-art on ACE05.
Modelling Context Emotions using Multi-task Learning for Emotion Controlled Dialog Generation
Authors: Deeksha Varshney, Asif Ekbal, Pushpak Bhattacharyya
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Decoder, Multi-Task Learning, Response Generation, Text Generation

A recent topic of research in natural language generation has been the development of automatic response generation modules that can automatically respond to a user{'}s utterance in an empathetic manner. Previous research has tackled this task using neural generative methods by augmenting emotion classes with the input sequences. However, the outputs by these models may be inconsistent. We employ multi-task learning to predict the emotion label and to generate a viable response for a given utterance using a common encoder with multiple decoders. Our proposed encoder-decoder model consists of a self-attention based encoder and a decoder with dot product attention mechanism to generate response with a specified emotion. We use the focal loss to handle imbalanced data distribution, and utilize the consistency loss to allow coherent decoding by the decoders. Human evaluation reveals that our model produces more emotionally pertinent responses. In addition, our model outperforms multiple strong baselines on automatic evaluation measures such as F1 and BLEU scores, thus resulting in more fluent and adequate responses.