pwc-archive/papers-with-abstracts huggingface dataset:
Understanding Procedural Text using Interactive Entity Networks
Authors: Jizhi Tang, Yansong Feng, Dongyan Zhao
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Reading Comprehension

The task of procedural text comprehension aims to understand the dynamic nature of entities/objects in a process. Here, the key is to track how the entities interact with each other and how their states are changing along the procedure. Recent efforts have made great progress to track multiple entities in a procedural text, but usually treat each entity separately and ignore the fact that there are often multiple entities interacting with each other during one process, some of which are even explicitly mentioned. In this paper, we propose a novel Interactive Entity Network (IEN), which is a recurrent network with memory equipped cells for state tracking. In each IEN cell, we maintain different attention matrices through specific memories to model different types of entity interactions. Importantly, we can update these memories in a sequential manner so as to explore the causal relationship between entity actions and subsequent state changes. We evaluate our model on a benchmark dataset, and the results show that IEN outperforms state-of-the-art models by precisely capturing the interactions of multiple entities and explicitly leverage the relationship between entity interactions and subsequent state changes.
Debiasing knowledge graph embeddings
Authors: Joseph Fisher, Arpit Mittal, Dave Palfrey, Christos Christodoulopoulos
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Knowledge Graph Embeddings

It has been shown that knowledge graph embeddings encode potentially harmful social biases, such as the information that women are more likely to be nurses, and men more likely to be bankers. As graph embeddings begin to be used more widely in NLP pipelines, there is a need to develop training methods which remove such biases. Previous approaches to this problem both significantly increase the training time, by a factor of eight or more, and decrease the accuracy of the model substantially. We present a novel approach, in which all embeddings are trained to be neutral to sensitive attributes such as gender by default using an adversarial loss. We then add sensitive attributes back on in whitelisted cases. Training time only marginally increases over a baseline model, and the debiased embeddings perform almost as accurately in the triple prediction task as their non-debiased counterparts.
Relation-aware Graph Attention Networks with Relational Position Encodings for Emotion Recognition in Conversations
Authors: Taichi Ishiwatari, Yuki Yasuda, Taro Miyazaki, Jun Goto
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Emotion Recognition, Emotion Recognition in Conversation, Graph Attention, Position, Relation

Interest in emotion recognition in conversations (ERC) has been increasing in various fields, because it can be used to analyze user behaviors and detect fake news. Many recent ERC methods use graph-based neural networks to take the relationships between the utterances of the speakers into account. In particular, the state-of-the-art method considers self- and inter-speaker dependencies in conversations by using relational graph attention networks (RGAT). However, graph-based neural networks do not take sequential information into account. In this paper, we propose relational position encodings that provide RGAT with sequential information reflecting the relational graph structure. Accordingly, our RGAT model can capture both the speaker dependency and the sequential information. Experiments on four ERC datasets show that our model is beneficial to recognizing emotions expressed in conversations. In addition, our approach empirically outperforms the state-of-the-art on all of the benchmark datasets.
Textual Data Augmentation for Efficient Active Learning on Tiny Datasets
Authors: Husam Quteineh, Spyridon Samothrakis, Richard Sutcliffe
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Active Learning, Data Augmentation, SST-2, Text Generation

In this paper we propose a novel data augmentation approach where guided outputs of a language generation model, e.g. GPT-2, when labeled, can improve the performance of text classifiers through an active learning process. We transform the data generation task into an optimization problem which maximizes the usefulness of the generated output, using Monte Carlo Tree Search (MCTS) as the optimization strategy and incorporating entropy as one of the optimization criteria. We test our approach against a Non-Guided Data Generation (NGDG) process that does not optimize for a reward function. Starting with a small set of data, our results show an increased performance with MCTS of 26{\%} on the TREC-6 Questions dataset, and 10{\%} on the Stanford Sentiment Treebank SST-2 dataset. Compared with NGDG, we are able to achieve increases of 3{\%} and 5{\%} on TREC-6 and SST-2.
``I'd rather just go to bed'': Understanding Indirect Answers
Authors: Annie Louis, Dan Roth, Filip Radlinski
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Language Modeling, Language Modelling, Transfer Learning

We revisit a pragmatic inference problem in dialog: Understanding indirect responses to questions. Humans can interpret {`}I{'}m starving.{'} in response to {`}Hungry?{'}, even without direct cue words such as {`}yes{'} and {`}no{'}. In dialog systems, allowing natural responses rather than closed vocabularies would be similarly beneficial. However, today{'}s systems are only as sensitive to these pragmatic moves as their language model allows. We create and release the first large-scale English language corpus {`}Circa{'} with 34,268 (polar question, indirect answer) pairs to enable progress on this task. The data was collected via elaborate crowdsourcing, and contains utterances with yes/no meaning, as well as uncertain, middle-ground, and conditional responses. We also present BERT-based neural models to predict such categories for a question-answer pair. We find that while transfer learning from entailment works reasonably, performance is not yet sufficient for robust dialog. Our models reach 82-88{\%} accuracy for a 4-class distinction, and 74-85{\%} for 6 classes.
Centering-based Neural Coherence Modeling with Hierarchical Discourse Segments
Authors: Sungho Jeon, Michael Strube
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Automated Essay Scoring, Language Modeling, Language Modelling, Sentence

Previous neural coherence models have focused on identifying semantic relations between adjacent sentences. However, they do not have the means to exploit structural information. In this work, we propose a coherence model which takes discourse structural information into account without relying on human annotations. We approximate a linguistic theory of coherence, Centering theory, which we use to track the changes of focus between discourse segments. Our model first identifies the focus of each sentence, recognized with regards to the context, and constructs the structural relationship for discourse segments by tracking the changes of the focus. The model then incorporates this structural information into a structure-aware transformer. We evaluate our model on two tasks, automated essay scoring and assessing writing quality. Our results demonstrate that our model, built on top of a pretrained language model, achieves state-of-the-art performance on both tasks. We next statistically examine the identified trees of texts assigned to different quality scores. Finally, we investigate what our model learns in terms of theoretical claims.
A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media
Authors: Ramit Sawhney, Harshit Joshi, Saumya Gandhi, Rajiv Ratn Shah
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11

Social media{'}s ubiquity fosters a space for users to exhibit suicidal thoughts outside of traditional clinical settings. Understanding the build-up of such ideation is critical for the identification of at-risk users and suicide prevention. Suicide ideation is often linked to a history of mental depression. The emotional spectrum of a user{'}s historical activity on social media can be indicative of their mental state over time. In this work, we focus on identifying suicidal intent in English tweets by augmenting linguistic models with historical context. We propose STATENet, a time-aware transformer based model for preliminary screening of suicidal risk on social media. STATENet outperforms competitive methods, demonstrating the utility of emotional and temporal contextual cues for suicide risk assessment. We discuss the empirical, qualitative, practical, and ethical aspects of STATENet for suicide ideation detection.
Constrained Fact Verification for FEVER
Authors: Adithya Pratapa, Sai Muralidhar Jayanthi, Kavya Nerella
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Fact Verification

Fact-verification systems are well explored in the NLP literature with growing attention owing to shared tasks like FEVER. Though the task requires reasoning on extracted evidence to verify a claim{'}s factuality, there is little work on understanding the reasoning process. In this work, we propose a new methodology for fact-verification, specifically FEVER, that enforces a closed-world reliance on extracted evidence. We present an extensive evaluation of state-of-the-art verification models under these constraints.
PatchBERT: Just-in-Time, Out-of-Vocabulary Patching
Authors: Sangwhan Moon, Naoaki Okazaki
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Transfer Learning

Large scale pre-trained language models have shown groundbreaking performance improvements for transfer learning in the domain of natural language processing. In our paper, we study a pre-trained multilingual BERT model and analyze the OOV rate on downstream tasks, how it introduces information loss, and as a side-effect, obstructs the potential of the underlying model. We then propose multiple approaches for mitigation and demonstrate that it improves performance with the same parameter count when combined with fine-tuning.
Active Learning for BERT: An Empirical Study
Authors: Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Active Learning, Binary text classification, Classification, General Classification, text-classification, Text Classification

Real world scenarios present a challenge for text classification, since labels are usually expensive and the data is often characterized by class imbalance. Active Learning (AL) is a ubiquitous paradigm to cope with data scarcity. Recently, pre-trained NLP models, and BERT in particular, are receiving massive attention due to their outstanding performance in various NLP tasks. However, the use of AL with deep pre-trained models has so far received little consideration. Here, we present a large-scale empirical study on active learning techniques for BERT-based classification, addressing a diverse set of AL strategies and datasets. We focus on practical scenarios of binary text classification, where the annotation budget is very small, and the data is often skewed. Our results demonstrate that AL can boost BERT performance, especially in the most realistic scenario in which the initial set of labeled examples is created using keyword-based queries, resulting in a biased sample of the minority class. We release our research framework, aiming to facilitate future research along the lines explored here.
VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls
Authors: Ramit Sawhney, Piyush Khanna, Arshiya Aggarwal, Taru Jain, Puneet Mathur, Rajiv Ratn Shah
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11

Natural language processing has recently made stock movement forecasting and volatility forecasting advances, leading to improved financial forecasting. Transcripts of companies{'} earnings calls are well studied for risk modeling, offering unique investment insight into stock performance. However, vocal cues in the speech of company executives present an underexplored rich source of natural language data for estimating financial risk. Additionally, most existing approaches ignore the correlations between stocks. Building on existing work, we introduce a neural model for stock volatility prediction that accounts for stock interdependence via graph convolutions while fusing verbal, vocal, and financial features in a semi-supervised multi-task risk forecasting formulation. Our proposed model, VolTAGE, outperforms existing methods demonstrating the effectiveness of multimodal learning for volatility prediction.
Effectively pretraining a speech translation decoder with Machine Translation data
Authors: Ashkan Alinejad, Anoop Sarkar
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Machine Translation, NMT, speech-recognition, Speech Recognition, Speech-to-Text, Translation

Directly translating from speech to text using an end-to-end approach is still challenging for many language pairs due to insufficient data. Although pretraining the encoder parameters using the Automatic Speech Recognition (ASR) task improves the results in low resource settings, attempting to use pretrained parameters from the Neural Machine Translation (NMT) task has been largely unsuccessful in previous works. In this paper, we will show that by using an adversarial regularizer, we can bring the encoder representations of the ASR and NMT tasks closer even though they are in different modalities, and how this helps us effectively use a pretrained NMT decoder for speech translation.
A Preliminary Exploration of GANs for Keyphrase Generation
Authors: Avinash Swaminathan, Haimin Zhang, Debanjan Mahata, Rakesh Gosangi, Rajiv Ratn Shah, Amanda Stent
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Keyphrase Generation

We introduce a new keyphrase generation approach using Generative Adversarial Networks (GANs). For a given document, the generator produces a sequence of keyphrases, and the discriminator distinguishes between human-curated and machine-generated keyphrases. We evaluated this approach on standard benchmark datasets. We observed that our model achieves state-of-the-art performance in the generation of abstractive keyphrases and is comparable to the best performing extractive techniques. Although we achieve promising results using GANs, they are not significantly better than the state-of-the-art generative models. To our knowledge, this is one of the first works that use GANs for keyphrase generation. We present a detailed analysis of our observations and expect that these findings would help other researchers to further study the use of GANs for the task of keyphrase generation.
TESA: A Task in Entity Semantic Aggregation for Abstractive Summarization
Authors: Cl{\'e}ment Jumel, Annie Louis, Jackie Chi Kit Cheung
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Abstractive Text Summarization, Decoder, Language Modeling, Language Modelling

Human-written texts contain frequent generalizations and semantic aggregation of content. In a document, they may refer to a pair of named entities such as {`}London{'} and {`}Paris{'} with different expressions: {``}the major cities{''}, {``}the capital cities{''} and {``}two European cities{''}. Yet generation, especially, abstractive summarization systems have so far focused heavily on paraphrasing and simplifying the source content, to the exclusion of such semantic abstraction capabilities. In this paper, we present a new dataset and task aimed at the semantic aggregation of entities. TESA contains a dataset of 5.3K crowd-sourced entity aggregations of Person, Organization, and Location named entities. The aggregations are document-appropriate, meaning that they are produced by annotators to match the situational context of a given news article from the New York Times. We then build baseline models for generating aggregations given a tuple of entities and document context. We finetune on TESA an encoder-decoder language model and compare it with simpler classification methods based on linguistically informed features. Our quantitative and qualitative evaluations show reasonable performance in making a choice from a given list of expressions, but free-form expressions are understandably harder to generate and evaluate.
Intrinsic Evaluation of Summarization Datasets
Authors: Rishi Bommasani, Claire Cardie
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11

High quality data forms the bedrock for building meaningful statistical models in NLP. Consequently, data quality must be evaluated either during dataset construction or *post hoc*. Almost all popular summarization datasets are drawn from natural sources and do not come with inherent quality assurance guarantees. In spite of this, data quality has gone largely unquestioned for many of these recent datasets. We perform the first large-scale evaluation of summarization datasets by introducing 5 intrinsic metrics and applying them to 10 popular datasets. We find that data usage in recent summarization research is sometimes inconsistent with the underlying properties of the data. Further, we discover that our metrics can serve the additional purpose of being inexpensive heuristics for detecting generically low quality examples.
Iterative Feature Mining for Constraint-Based Data Collection to Increase Data Diversity and Model Robustness
Authors: Stefan Larson, Anthony Zheng, Anish Mahendran, Rishi Tekriwal, Adrian Cheung, Eric Guldan, Kevin Leach, Jonathan K. Kummerfeld
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Diversity, intent-classification, Intent Classification, Intent Classification and Slot Filling, slot-filling, Slot Filling

Diverse data is crucial for training robust models, but crowdsourced text often lacks diversity as workers tend to write simple variations from prompts. We propose a general approach for guiding workers to write more diverse text by iteratively constraining their writing. We show how prior workflows are special cases of our approach, and present a way to apply the approach to dialog tasks such as intent classification and slot-filling. Using our method, we create more challenging versions of test sets from prior dialog datasets and find dramatic performance drops for standard models. Finally, we show that our approach is complementary to recent work on improving data diversity, and training on data collected with our approach leads to more robust models.
Interview: Large-scale Modeling of Media Dialog with Discourse Patterns and Knowledge Grounding
Authors: Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, Julian McAuley
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11

In this work, we perform the first large-scale analysis of discourse in media dialog and its impact on generative modeling of dialog turns, with a focus on interrogative patterns and use of external knowledge. Discourse analysis can help us understand modes of persuasion, entertainment, and information elicitation in such settings, but has been limited to manual review of small corpora. We introduce **Interview**{---}a large-scale (105K conversations) media dialog dataset collected from news interview transcripts{---}which allows us to investigate such patterns at scale. We present a dialog model that leverages external knowledge as well as dialog acts via auxiliary losses and demonstrate that our model quantitatively and qualitatively outperforms strong discourse-agnostic baselines for dialog modeling{---}generating more specific and topical responses in interview-style conversations.
Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies
Authors: Maryam Aminian, Mohammad Sadegh Rasooli, Mona Diab
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Cross-Lingual Transfer

We describe a method for developing broad-coverage semantic dependency parsers for languages for which no semantically annotated resource is available. We leverage a multitask learning framework coupled with annotation projection. We use syntactic parsing as the auxiliary task in our multitask setup. Our annotation projection experiments from English to Czech show that our multitask setup yields 3.1{\%} (4.2{\%}) improvement in labeled F1-score on in-domain (out-of-domain) test set compared to a single-task baseline.
Text Graph Transformer for Document Classification
Authors: Haopeng Zhang, Jiawei Zhang
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Classification, Document Classification, General Classification, Graph Neural Network, Graph Sampling, text-classification, Text Classification

Text classification is a fundamental problem in natural language processing. Recent studies applied graph neural network (GNN) techniques to capture global word co-occurrence in a corpus. However, previous works are not scalable to large-sized corpus and ignore the heterogeneity of the text graph. To address these problems, we introduce a novel Transformer based heterogeneous graph neural network, namely Text Graph Transformer (TG-Transformer). Our model learns effective node representations by capturing structure and heterogeneity from the text graph. We propose a mini-batch text graph sampling method that significantly reduces computing and memory costs to handle large-sized corpus. Extensive experiments have been conducted on several benchmark datasets, and the results demonstrate that TG-Transformer outperforms state-of-the-art approaches on text classification task.
META: Metadata-Empowered Weak Supervision for Text Classification
Authors: Dheeraj Mekala, Xinyang Zhang, Jingbo Shang
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Classification, General Classification, text-classification, Text Classification, Weakly-supervised Learning

Recent advances in weakly supervised learning enable training high-quality text classifiers by only providing a few user-provided seed words. Existing methods mainly use text data alone to generate pseudo-labels despite the fact that metadata information (e.g., author and timestamp) is widely available across various domains. Strong label indicators exist in the metadata and it has been long overlooked mainly due to the following challenges: (1) metadata is multi-typed, requiring systematic modeling of different types and their combinations, (2) metadata is noisy, some metadata entities (e.g., authors, venues) are more compelling label indicators than others. In this paper, we propose a novel framework, META, which goes beyond the existing paradigm and leverages metadata as an additional source of weak supervision. Specifically, we organize the text data and metadata together into a text-rich network and adopt network motifs to capture appropriate combinations of metadata. Based on seed words, we rank and filter motif instances to distill highly label-indicative ones as {``}seed motifs{''}, which provide additional weak supervision. Following a bootstrapping manner, we train the classifier and expand the seed words and seed motifs iteratively. Extensive experiments and case studies on real-world datasets demonstrate superior performance and significant advantages of leveraging metadata as weak supervision.