pwc-archive/papers-with-abstracts huggingface dataset:
Automated Pyramid Summarization Evaluation
Authors: Yanjun Gao, Chen Sun, Rebecca J. Passonneau
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11

Pyramid evaluation was developed to assess the content of paragraph length summaries of source texts. A pyramid lists the distinct units of content found in several reference summaries, weights content units by how many reference summaries they occur in, and produces three scores based on the weighted content of new summaries. We present an automated method that is more efficient, more transparent, and more complete than previous automated pyramid methods. It is tested on a new dataset of student summaries, and historical NIST data from extractive summarizers.
Leveraging Past References for Robust Language Grounding
Authors: Subhro Roy, Michael Noseworthy, Rohan Paul, Daehyung Park, Nicholas Roy
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Object, Referring Expression, Visual Grounding

Grounding referring expressions to objects in an environment has traditionally been considered a one-off, ahistorical task. However, in realistic applications of grounding, multiple users will repeatedly refer to the same set of objects. As a result, past referring expressions for objects can provide strong signals for grounding subsequent referring expressions. We therefore reframe the grounding problem from the perspective of coreference detection and propose a neural network that detects when two expressions are referring to the same object. The network combines information from vision and past referring expressions to resolve which object is being referred to. Our experiments show that detecting referring expression coreference is an effective way to ground objects described by subtle visual properties, which standard visual grounding models have difficulty capturing. We also show the ability to detect object coreference allows the grounding model to perform well even when it encounters object categories not seen in the training data.
Active Learning via Membership Query Synthesis for Semi-Supervised Sentence Classification
Authors: Raphael Schumann, Ines Rehbein
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Active Learning, General Classification, Sentence, Sentence Classification, text-classification, Text Classification

Active learning (AL) is a technique for reducing manual annotation effort during the annotation of training data for machine learning classifiers. For NLP tasks, pool-based and stream-based sampling techniques have been used to select new instances for AL while gen erating new, artificial instances via Membership Query Synthesis was, up to know, considered to be infeasible for NLP problems. We present the first successfull attempt to use Membership Query Synthesis for generating AL queries, using Variational Autoencoders for query generation. We evaluate our approach in a text classification task and demonstrate that query synthesis shows competitive performance to pool-based AL strategies while substantially reducing annotation time
A General-Purpose Algorithm for Constrained Sequential Inference
Authors: Daniel Deutsch, Shyam Upadhyay, Dan Roth
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Constituency Parsing, Semantic Role Labeling, Structured Prediction, valid

Inference in structured prediction involves finding the best output structure for an input, subject to certain constraints. Many current approaches use sequential inference, which constructs the output in a left-to-right manner. However, there is no general framework to specify constraints in these approaches. We present a principled approach for incorporating constraints into sequential inference algorithms. Our approach expresses constraints using an automaton, which is traversed in lock-step during inference, guiding the search to valid outputs. We show that automata can express commonly used constraints and are easily incorporated into sequential inference. When it is more natural to represent constraints as a set of automata, our algorithm uses an active set method for demonstrably fast and efficient inference. We experimentally show the benefits of our algorithm on constituency parsing and semantic role labeling. For parsing, unlike unconstrained approaches, our algorithm always generates valid output, incurring only a small drop in performance. For semantic role labeling, imposing constraints using our algorithm corrects common errors, improving F1 by 1.5 points. These benefits increase in low-resource settings. Our active set method achieves a 5.2x relative speed-up over a naive approach.
Detecting Frames in News Headlines and Its Application to Analyzing News Framing Trends Surrounding U.S. Gun Violence
Authors: Siyi Liu, Lei Guo, Kate Mays, Margrit Betke, Derry Tanti Wijaya
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Articles

Different news articles about the same topic often offer a variety of perspectives: an article written about gun violence might emphasize gun control, while another might promote 2nd Amendment rights, and yet a third might focus on mental health issues. In communication research, these different perspectives are known as {``}frames{''}, which, when used in news media will influence the opinion of their readers in multiple ways. In this paper, we present a method for effectively detecting frames in news headlines. Our training and performance evaluation is based on a new dataset of news headlines related to the issue of gun violence in the United States. This Gun Violence Frame Corpus (GVFC) was curated and annotated by journalism and communication experts. Our proposed approach sets a new state-of-the-art performance for multiclass news frame detection, significantly outperforming a recent baseline by 35.9{\%} absolute difference in accuracy. We apply our frame detection approach in a large scale study of 88k news headlines about the coverage of gun violence in the U.S. between 2016 and 2018.
KnowSemLM: A Knowledge Infused Semantic Language Model
Authors: Haoruo Peng, Qiang Ning, Dan Roth
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Cloze Test, Language Modeling, Language Modelling, model

Story understanding requires developing expectations of what events come next in text. Prior knowledge {--} both statistical and declarative {--} is essential in guiding such expectations. While existing semantic language models (SemLM) capture event co-occurrence information by modeling event sequences as semantic frames, entities, and other semantic units, this paper aims at augmenting them with causal knowledge (i.e., one event is likely to lead to another). Such knowledge is modeled at the frame and entity level, and can be obtained either statistically from text or stated declaratively. The proposed method, KnowSemLM, infuses this knowledge into a semantic LM by joint training and inference, and is shown to be effective on both the event cloze test and story/referent prediction tasks.
Roll Call Vote Prediction with Knowledge Augmented Models
Authors: Pallavi Patil, Kriti Myer, Ronak Zala, Arpit Singh, Sheshera Mysore, Andrew Mccallum, Adrian Benton, Am Stent, a
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Prediction

The official voting records of United States congresspeople are preserved as roll call votes. Prediction of voting behavior of politicians for whom no voting record exists, such as individuals running for office, is important for forecasting key political decisions. Prior work has relied on past votes cast to predict future votes, and thus fails to predict voting patterns for politicians without voting records. We address this by augmenting a prior state of the art model with multiple sources of external knowledge so as to enable prediction on unseen politicians. The sources of knowledge we use are news text and Freebase, a manually curated knowledge base. We propose augmentations based on unigram features for news text, and a knowledge base embedding method followed by a neural network composition for relations from Freebase. Empirical evaluation of these approaches indicate that the proposed models outperform the prior system for politicians with complete historical voting records by 1.0{\%} point of accuracy (8.7{\%} error reduction) and for politicians without voting records by 33.4{\%} points of accuracy (66.7{\%} error reduction). We also show that the knowledge base augmented approach outperforms the news text augmented approach by 4.2{\%} points of accuracy.
BeamSeg: A Joint Model for Multi-Document Segmentation and Topic Identification
Authors: Pedro Mota, Maxine Eskenazi, Lu{\'\i}sa Coheur
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Language Modeling, Language Modelling, Segmentation

We propose BeamSeg, a joint model for segmentation and topic identification of documents from the same domain. The model assumes that lexical cohesion can be observed across documents, meaning that segments describing the same topic use a similar lexical distribution over the vocabulary. The model implements lexical cohesion in an unsupervised Bayesian setting by drawing from the same language model segments with the same topic. Contrary to previous approaches, we assume that language models are not independent, since the vocabulary changes in consecutive segments are expected to be smooth and not abrupt. We achieve this by using a dynamic Dirichlet prior that takes into account data contributions from other topics. BeamSeg also models segment length properties of documents based on modality (textbooks, slides, \textit{etc.}). The evaluation is carried out in three datasets. In two of them, improvements of up to 4.8{\%} and 7.3{\%} are obtained in the segmentation and topic identifications tasks, indicating that both tasks should be jointly modeled.
MrMep: Joint Extraction of Multiple Relations and Multiple Entity Pairs Based on Triplet Attention
Authors: Jiayu Chen, Caixia Yuan, Xiaojie Wang, Ziwei Bai
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Decoder, Relation, Triplet

This paper focuses on how to extract multiple relational facts from unstructured text. Neural encoder-decoder models have provided a viable new approach for jointly extracting relations and entity pairs. However, these models either fail to deal with entity overlapping among relational facts, or neglect to produce the whole entity pairs. In this work, we propose a novel architecture that augments the encoder and decoder in two elegant ways. First, we apply a binary CNN classifier for each relation, which identifies all possible relations maintained in the text, while retaining the target relation representation to aid entity pair recognition. Second, we perform a multi-head attention over the text and a triplet attention with the target relation interacting with every token of the text to precisely produce all possible entity pairs in a sequential manner. Experiments on three benchmark datasets show that our proposed method successfully addresses the multiple relations and multiple entity pairs even with complex overlapping and significantly outperforms the state-of-the-art methods.
Exploiting the Entity Type Sequence to Benefit Event Detection
Authors: Yuze Ji, Youfang Lin, Jianwei Gao, Huaiyu Wan
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Event Detection, Vocal Bursts Type Prediction

Event Detection (ED) is one of the most important task in the field of information extraction. The goal of ED is to find triggers in sentences and classify them into different event types. In previous works, the information of entity types are commonly utilized to benefit event detection. However, the sequential features of entity types have not been well utilized yet in the existing ED methods. In this paper, we propose a novel ED approach which learns sequential features from word sequences and entity type sequences separately, and combines these two types of sequential features with the help of a trigger-entity interaction learning module. The experimental results demonstrate that our proposed approach outperforms the state-of-the-art methods.
Named Entity Recognition - Is There a Glass Ceiling?
Authors: Tomasz Stanislawek, Anna Wr{\'o}blewska, Alicja W{\'o}jcicka, Daniel Ziembicki, Przemyslaw Biecek
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: BIG-bench Machine Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Recent developments in Named Entity Recognition (NER) have resulted in better and better models. However, is there a glass ceiling? Do we know which types of errors are still hard or even impossible to correct? In this paper, we present a detailed analysis of the types of errors in state-of-the-art machine learning (ML) methods. Our study illustrates weak and strong points of the Stanford, CMU, FLAIR, ELMO and BERT models, as well as their shared limitations. We also introduce new techniques for improving annotation, training process, and for checking model quality and stability.
Low-Rank Approximations of Second-Order Document Representations
Authors: Jarkko Lagus, Janne Sinkkonen, Arto Klami
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Articles, Sentence

Document embeddings, created with methods ranging from simple heuristics to statistical and deep models, are widely applicable. Bag-of-vectors models for documents include the mean and quadratic approaches (Torki, 2018). We present evidence that quadratic statistics alone, without the mean information, can offer superior accuracy, fast document comparison, and compact document representations. In matching news articles to their comment threads, low-rank representations of only 3-4 times the size of the mean vector give most accurate matching, and in standard sentence comparison tasks, results are state of the art despite faster computation. Similarity measures are discussed, and the Frobenius product implicit in the proposed method is contrasted to Wasserstein or Bures metric from the transportation theory. We also shortly demonstrate matching of unordered word lists to documents, to measure topicality or sentiment of documents.
Contextualized Cross-Lingual Event Trigger Extraction with Minimal Resources
Authors: Meryem M{'}hamdi, Marjorie Freedman, Jonathan May
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Event Extraction, Transfer Learning

Event trigger extraction is an information extraction task of practical utility, yet it is challenging due to the difficulty of disambiguating word sense meaning. Previous approaches rely extensively on hand-crafted language-specific features and are applied mainly to English for which annotated datasets and Natural Language Processing (NLP) tools are available. However, the availability of such resources varies from one language to another. Recently, contextualized Bidirectional Encoder Representations from Transformers (BERT) models have established state-of-the-art performance for a variety of NLP tasks. However, there has not been much effort in exploring language transfer using BERT for event extraction. In this work, we treat event trigger extraction as a sequence tagging problem and propose a cross-lingual framework for training it without any hand-crafted features. We experiment with different flavors of transfer learning from high-resourced to low-resourced languages and compare the performance of different multilingual embeddings for event trigger extraction. Our results show that training in a multilingual setting outperforms language-specific models for both English and Chinese. Our work is the first to experiment with two event architecture variants in a cross-lingual setting, to show the effectiveness of contextualized embeddings obtained using BERT, and to explore and analyze its performance on Arabic.
SimVecs: Similarity-Based Vectors for Utterance Representation in Conversational AI Systems
Authors: Ashraf Mahgoub, Youssef Shahin, Riham Mansour, Saurabh Bagchi
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Sentence

Conversational AI systems are gaining a lot of attention recently in both industrial and scientific domains, providing a natural way of interaction between customers and adaptive intelligent systems. A key requirement in these systems is the ability to understand the user{'}s intent and provide adequate responses to them. One of the greatest challenges of language understanding (LU) services is efficient utterance (sentence) representation in vector space, which is an essential step for most ML tasks. In this paper, we propose a novel approach for generating vector space representations of utterances using pair-wise similarity metrics. The proposed approach uses only a few corpora to tune the weights of the similarity metric without relying on external general purpose ontologies. Our experiments confirm that the generated vectors can improve the performance of LU services in unsupervised, semi-supervised and supervised learning tasks.
Memory Graph Networks for Explainable Memory-grounded Question Answering
Authors: Seungwhan Moon, Pararth Shah, Anuj Kumar, Rajen Subba
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Question Answering

We introduce Episodic Memory QA, the task of answering personal user questions grounded on memory graph (MG), where episodic memories and related entity nodes are connected via relational edges. We create a new benchmark dataset first by generating synthetic memory graphs with simulated attributes, and by composing 100K QA pairs for the generated MG with bootstrapped scripts. To address the unique challenges for the proposed task, we propose Memory Graph Networks (MGN), a novel extension of memory networks to enable dynamic expansion of memory slots through graph traversals, thus able to answer queries in which contexts from multiple linked episodes and external knowledge are required. We then propose the Episodic Memory QA Net with multiple module networks to effectively handle various question types. Empirical results show improvement over the QA baselines in top-k answer prediction accuracy in the proposed task. The proposed model also generates a graph walk path and attention vectors for each predicted answer, providing a natural way to explain its QA reasoning.
Relation Module for Non-Answerable Predictions on Reading Comprehension
Authors: Kevin Huang, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Machine Reading Comprehension, Reading Comprehension, Relation, Relation Network, Sentence

Machine reading comprehension (MRC) has attracted significant amounts of research attention recently, due to an increase of challenging reading comprehension datasets. In this paper, we aim to improve a MRC model{'}s ability to determine whether a question has an answer in a given context (e.g. the recently proposed SQuAD 2.0 task). The relation module consists of both semantic extraction and relational information. We first extract high level semantics as objects from both question and context with multi-head self-attentive pooling. These semantic objects are then passed to a relation network, which generates relationship scores for each object pair in a sentence. These scores are used to determine whether a question is non-answerable. We test the relation module on the SQuAD 2.0 dataset using both the BiDAF and BERT models as baseline readers. We obtain 1.8{\%} gain of F1 accuracy on top of the BiDAF reader, and 1.0{\%} on top of the BERT base model. These results show the effectiveness of our relation module on MRC.
Slot Tagging for Task Oriented Spoken Language Understanding in Human-to-Human Conversation Scenarios
Authors: Kunho Kim, Rahul Jha, Kyle Williams, Alex Marin, Imed Zitouni
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Spoken Language Understanding

Task oriented language understanding (LU) in human-to-machine (H2M) conversations has been extensively studied for personal digital assistants. In this work, we extend the task oriented LU problem to human-to-human (H2H) conversations, focusing on the slot tagging task. Recent advances on LU in H2M conversations have shown accuracy improvements by adding encoded knowledge from different sources. Inspired by this, we explore several variants of a bidirectional LSTM architecture that relies on different knowledge sources, such as Web data, search engine click logs, expert feedback from H2M models, as well as previous utterances in the conversation. We also propose ensemble techniques that aggregate these different knowledge sources into a single model. Experimental evaluation on a four-turn Twitter dataset in the restaurant and music domains shows improvements in the slot tagging F1-score of up to 6.09{\%} compared to existing approaches.
Window-Based Neural Tagging for Shallow Discourse Argument Labeling
Authors: Ren{\'e} Knaebel, Manfred Stede, Sebastian Stober
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Discourse Parsing

This paper describes a novel approach for the task of end-to-end argument labeling in shallow discourse parsing. Our method describes a decomposition of the overall labeling task into subtasks and a general distance-based aggregation procedure. For learning these subtasks, we train a recurrent neural network and gradually replace existing components of our baseline by our model. The model is trained and evaluated on the Penn Discourse Treebank 2 corpus. While it is not as good as knowledge-intense approaches, it clearly outperforms other models that are also trained without additional linguistic features.
TILM: Neural Language Models with Evolving Topical Influence
Authors: Shubhra Kanti Karmaker Santu, Kalyan Veeramachaneni, ChengXiang Zhai
Date: 2019-11-01 00:00:00
Proceeding: CONLL 2019 11
Tasks: Language Modeling, Language Modelling

Content of text data are often influenced by contextual factors which often evolve over time (e.g., content of social media are often influenced by topics covered in the major news streams). Existing language models do not consider the influence of such related evolving topics, and thus are not optimal. In this paper, we propose to incorporate such topical-influence into a language model to both improve its accuracy and enable cross-stream analysis of topical influences. Specifically, we propose a novel language model called Topical Influence Language Model (TILM), which is a novel extension of a neural language model to capture the influences on the contents in one text stream by the evolving topics in another related (or possibly same) text stream. Experimental results on six different text stream data comprised of conference paper titles show that the incorporation of evolving topical influence into a language model is beneficial and TILM outperforms multiple baselines in a challenging task of text forecasting. In addition to serving as a language model, TILM further enables interesting analysis of topical influence among multiple text streams.